Personalised welcome message via facial recognition

29 mei 2019 Inzichten /

As featured on the Medium platform on the 27th of May - Written by Stijn Spanhove.
At Wijs we organise a Trend Event for our customers every year. But this time, we wanted to do something special. It was our goal to show the visitors how well facial recognition works, and how easily it can be implemented with a limited set of data. The idea was to give the attendees a personalised welcome message when entering the office.


Face recognition

We decided to work with this Python library. They built a deep learning model that recognizes face landmarks on a photo. Next, it calculates the distance between those landmarks. The resulting figures are then compared with the landmarks on the attendee’s profile picture we found on the internet.


Landmarks on the face rendered

You’d be surprised how easily you can find a usable reference photo to practice face recognition. Nowadays, basically everyone has a suitable profile picture. Even if that photo is quite small, the only thing we needed were the face landmarks.

Our Python script processed the video feed from an IP camera frame by frame. If someone was recognised, a request was sent to our CMS (Harmony).

A visual representation of our complete setup

Harmony

Harmony is our tailor-made CMS based on Symfony. Since there are some core components already available, we could set up a CMS very fast. The account managers could easily add attendees and write a personalised welcome message.

We also made it possible to manage the templates used on the TV screens and dynamically assign available slots for personal messages.

Manage templates and slots of the screens in Harmony

Socket.IO

We’ve used the browser of the TV to run our web application. Via WebSockets and a socket.io server we were able to display the personal message in realtime. Every TV screen was connected and had available ’slots’. Our application was able to fill these slots with a message. After 10 seconds the message disappeared and the slot was available for a new message.

If a person was recognised we sent a request to our CMS and the CMS fetched the personal message from the database and sent it to a free slot on the TV. We’ve put some extra work into it to make this process as fast as possible. From recognizing the visitor’s face to seeing the response on TV, we aimed for 2 seconds.




Conclusion

I was quite surprised how well everything worked. There were, however, some false positives but that was mostly because of the poor quality of certain pictures.

This made me think. If we can create this with limited assets, what would people with bad intentions be able to do? You could download profile pictures from Facebook very easily, upload it into the system and use your smartphone camera without anyone even noticing it? Stuff to think about…

Shout-out to the Gents team that created the nice visuals and Koen who created this wst-block-16ith me.



Febelfin Academy

In the meantime we’ve also implemented this setup for the annual event of Febelfin Academy in the museum of the National Bank of Belgium!

Ook zin om mee te werken aan zo'n cool project?

We zoeken nog volk!

Stijn Spanhove
PHP Developer

Lees meer

Nieuwsbrief

Doe zoals meer dan 1700 marketing en design experts en ontvang maandelijks onze nieuwsbrief vol inzichten, tips en verrassende nieuwigheden.