Door unlocking using face recognition with IoT and Azure Cognitive Services
The customer, a shared space provider in major metro cities, provides access to its boarders into their accommodation using RFID based scanner, where the boarders have to carry their access card along with them all the time in order to have continued access. The following were the issues, faced by the customer due to the use of RFID
- In case, the boarder loses the RFID card, he will have to contact the location administrator to get a new card.
- The time between the loss of card and procuring of the new card, the boarder’s entry exit is not recorded
- In case the boarder forgets the card, again his entry exits cannot be tracked
- A card has to be provided for each user which is an additional cost to the customer
- The mis-use and mis-appropriation of RFID is highly likely since anybody with the access card will have entry into the property
In order to provide a fool proof solution to the customer problems, we at Paripoorna proposed them with a cutting edge solution of using face recognition for unlocking the doors using Azure’s Cognitive services and Internet of Things. In the customer’s building, we secured the entry using Face Recognition where the entry is logged and monitored for secured access and attendance logging.
- Azure cognitive services
- Azure Face Recognition
- Machine Learning
- Secure TCP / IP
- Android OS
- Java / Android
- Encrypted JSON
- Secure TCP / IP
It is a single board microcontroller which is based on ESP-12 module
- Tensilica Xtensa LX106
- XTOS operating system
- ESP8266 Wi-Fi Chip
- GCC Toolchain
- 128 KB Ram / 4 MB Flash storage
- Relay with Optocoupler
Enabled with performance level standards to meet the optimal requirements of the solution
- 802.11 b/g/n/e/i @ 2.4-2.5GHz support
- Wi-Fi Direct
- P2P with Group Owner, Group Client module
- Power management for specific mode
- WPA/WPA2 PSK and WPS driver for Secure connectivity
- WEP/TKIP/AES for encryption
- Open Interfaces with authentication
- IPv4, TCP/UDP/HTTP/FTP/MQTT Network protocol support
- The Architecture was studied for deployment in terms of scalability, requirements were listed out.
- Security was considered as an important parameter. This was done by developing a random encrypted string and adding a payload acceptor on device.
- Encrypted SSL/TLS certificate was developed with the firmware for secure payload transfer.
- After a careful analysis of all the wireless transfer protocols and wireless technologies available to develop a secure IoT, we finalized on using WLAN for feasibility and easy access to the user facing tab application.
- The TCP/IP protocol was preferred to ensure seamless connection between the endpoint and IoT Device through SSL/TLS for secure transfer.
- An API was developed in the local environment which could be accessed remotely for sending the image of a person as a request and to perform actions like detecting face in the image and sending the data corresponding to the image.
- This API was backed up by the powerful Azure cognitive services.
- The application sends a request to an IoT Device. On sending request with proper encrypted payload toggles the lock for 4 seconds and automatically switches the door ‘ON’.
- This method of processing makes it easy to integrate with any kind of user facing application that is able to share an API through the same wireless LAN.
- Then on completion for the access of entry point, circuit had to be redesigned to create an exit button, which was designed with an industrial grade, extremely sensitive PUSH button for complete user experience.
- For registering a user’s face and details into our application, the face image and the corresponding details are sent to an API which recognizes the face from the image and generates an Id and this Id is stored against the details of the user in the database.
- There has been a scenario where the access must be revoked for certain users. This is customized using our web application to revoke and provide access.
- An admin panel was developed were the administrators will be able to allow and revoke access if not authorized, which are categorized based on the description by the administrative level, Providing access based on their scope of requirement. Access can be revoked using panel from anywhere.
The below code snippet on the left, represents the code to control the lock and server payload control. In the right is the code to configure Wi-Fi and initial settings for boot up
- In case of users pressing the button multiple times causing the button to switch multiple times. This was handled using a built-in interrupt control.
- In the embedded part of the device, circuit was designed in such a way that the device itself was foolproof and designed for efficient usage. This ensured that the door was locked in case the device failure or reset.
- In case of loss of internet connection in router, the device had to be switched to another router. The switching the router at the tab end was easy whereas in IoT Device, it had to be programmed manually, this feature enabled seamless switching of device to any available Wi-Fi on the floor without the need for any customization.
- In order to improve the image quality in low lit and brightly lit conditions, image enhancement was done by incorporating contrast enlarging and de-noising in the app.
- There was an Issue where the image recognition and response time to unlock the door was a bit longer than expected, which was later recognized as due to image processing, which was later fixed by optimizing the image processing steps, there by exponentially reducing the response time.
- In scenarios, where multiple people had to enter into the office, so multiple faces had to be recognized. This was handled by customizing the application to recognize each and every person entering as a group in a single image by adding specific response for each and every face recognized in the group.
- In case of android application, the application was customized for interactive user experience, like screen savers with current time.
- Waving hand over the screen of the tab triggers image capture interface.
- Upon capturing the image, the application will retrieve the user’s details and give personalized notifications like birthday messages etc.
- Implementations where done to customize the user interface for a neat and clean welcome note on entry. For example, in case they’re laughing, our application will remind them to ‘Brighten someone’s day with this smile.’ or if not laughing, ‘Things may happen, but smile is the prettiest thing you could wear on!’
Conclusion & Business Impact
With this cutting edge solution we achieved the objective of
- Increased security ensuring the entry to the client’s property is restricted to only the registered users.
- Negates the chances any human error in tracking the user activities.
- Tracking not just the registered user but also the visitor ensures high end security.
- Biometric facial recognition technology can be easily integrated into the client’s current system
The IoT device used in the current architecture has the ability to scale up according to the number of users that will get registered into the system. Going further, Machine Learning can be used to do emotional analysis of the user in order to trigger appropriate notifications into their registered smart phones.
Tell us a little about yourself, and we'll be in touch right away