AI based surveillance system for monitoring key lock

The problem statement came from a real-life situation when data was not logged. A series of brainstorming sessions were held to discuss the problem. The conclusion was that inherent self-discipline is the reason for all the wrongdoings. The challenge was to make the users conscious that they are obliged to log the data. Before the implementation of the actual project, the users were multiple times asked to log the data. But we found no success since there wasn’t a way to check if the person obeyed the rules.

If the moment of the users is monitored, then there are possibilities that they become more aware of the rules. The face recognition technology came in handy this time. The solution should be straightforward for the administrative body to have data about the user moment, and the user should have this in the back of the mind to obey rules. A surveillance system after the consent of the users was developed.

The rough sketches were drawn to visualize the positions of the uses, camera, and key locker. The solution drifted a bit to add an embedded system but was dropped since it would increase unnecessary complexity. The final system was computer-based with all the features included. 

Flow:

1. The system detected faces only after the door was opened.
2. Once face detected, it was encoded to generate a face encoding vector of 128 values. 
3. The encodings are compared with the previously encoded list.
4. The encoding which has a face distance of less than 0.4 was assigned.
5. An ‘if statement’ suggesting that if the person is still present for more than 15 seconds. It checks to makes sure if the name isn’t enrolled unnecessarily.
6. For the first time users, it saves their name as well without overwriting 

There are three conditions to make sure the program doesn’t crash. It was challenging to meet the conditions since you can’t afford to crash the program. It was a tricky situation to think of code that performs all three times.

1. If there’s no name in the database: (First time user)

We are using the dictionary reading method to check the names. We pick the header and examine the header content that is name and time. Why do we do this is discussed briefly in the code explanation. It was crucial to make sure the file is checked for a name. 

2. If the name is already in the list:

The previous task was to verify if there’s a name in the list, if it’s isn’t then we have to save it. But here’s a catch, we cant save the name without comparing the last entry time.

3. Criteria for saving the name:

 It was the trickiest part of the code. First, we were required to check the name and the last entry time. The method I used was to check for a name and append all the time entries of that person. Once appended, the list was sorted to find the newest entry. Then the newest entry is compared with the current time. The latest time should be dynamically changing every passing second using a while loop in our conditions. 

I have assumed that it would take 15 seconds max for a person to get the key, and mention it in the log. It is used to avoid overpopulating the file with the same name for every second of a person’s presence. The current and the latest entry are subtracted in the loop and the difference is used to take the next decision. We will see how its actually works in the code explanation.

The code is the same in the basic structure with the face recognition project. The accuracy that we achieved in the previous project helps build a strong foundation for this project. The significant differences are the conditions we have adopted to make decisions for saving. If you prefer to study the entire code visit this link.

We first created a function called ‘markPresense’ for working our data. A file name after the present date is opened and we read the data hereafter(if there is no data in the file, an empty list would be returned). We also create a list to store the data in the future.

Note: we have saved header ‘Name’ and ‘Time’ after we generated the file.

We pass the data through a for loop and save the name and the time in two separate lists.

Here we are checking for the first condition. Is there the name we found after comparing faces is present in the file otherwise we save it.The current time is also saved in the desired format in the table.

What if the same person returns after taking the key a few times. We will have a bunch of names and time with decisions to make. What should be the criteria to save it for the next time. It is assumed that if the person returns after 15 seconds we are Sure shot going to save the name. But how would you find if the person is still present or moved?

To solve this we take only the present user’s name from the list. Find the newest entry, compare it with the latest time and check the condition. All this in a single block of code with multiple for, while loops, and if, else conditions.

Now we have the latest entry time we will compare it with the current time. The current time is found by using a While loop with a break at the end.

Here you can see how exactly the time is subtracted. It is important to have an idea about the strptime. Using it we convert a time string to datetime format.

The value helps you make the final decision about name saving. If it’s within the 15 seconds mark the name is not saved. It’s saved when the difference is more than needed.

It was a fun project which helped me realize the importance of core concepts. It was challenging to think in terms of the flow of the code. I learned the approach when working with a project in software development. I used Notion and particularly Kanban to keep track of the progress. Notion is a great tool to organize your data all in one place. 

With the limited time I had, I was able to develop it as per requirement. My recommendations for anyone who wishes to work on this project is :

1.  Run the project on the Linux platform. Later the project can be moved to Jetson by Nvidia (best for CV projects) to make the system modular.
2.  I was not able to autostart even after tweaking the BIOS settings Windows has issues with autorun. I hope Linux will solve the problem as proposed above.
3. Try out a bunch of other libraries, see how they work in the project. 
4. We encode faces every time we run the program. So think if you could create an encode list first, save it and later use it in the program. Thus saving on the starting time.