Software Development

SmartLuggage – The Application for Easy Packing

I built an app that makes a packing easy through an easy-to-use smart item allocation system. It allocates items to the appropriate category based on the user's activity and gender. It calculates the weight and number of items in the packed luggage. It allows you to add your own items and delete unnecessary ones. The colors in the app are well thought out and consistent. Google Map API is used to search destination, it allows to find practically every address in the world that is in Google database. Weather API is used to displays the temperature and an icon with the current weather status. The vision system allows the user to add elements using a camera. Application uses a cocoSSD model and Tensorflow.js. User can switch between cameras available on the device. The application is intuitive and uses the latest technology. In addition registration, login and administration panel is implemented.

The Frontend is hosted on Google Cloud. Backend is pushed to the the Heroku. Can be viewed via this link. It can load slowly at first as I use low cost hosting and if the app is not used it goes to sleep.

More details

To run the application, I created a bash script as shown in the photo below.

The largest window is the main window. There is a smaller blue square in this window. This is the place where we need to place the object we want to recognize.The smaller window on the left shows only what we see in this blue zone.

Press the space bar to take the photo. New window will appear with the photo that just have been taken. The system will ask you if you are happy with the result. If so, confirm by pressing esc or if not press space again and choose another frame.

The image will then be saved to your hard drive and object recognition will begin.

The other two objects stored in the database are a G-Shock watch and a pack of Oral-B toothpaste. They are displayed on the screen on the top left corner If you want to test the app, you can open the photo stored in the app and take a picture with your phone, then show the photo instead of the object - in this case the app works much more accurately.

Python code for capture the photo from the user.

                                         
            #camera on
            import cv2
            cam = cv2.VideoCapture(0)
            
            cv2.namedWindow("Object1")
            
            img_counter = 0
            
            #looping and waiting for user action
            while True:
                ret, frame = cam.read()
                if not ret:
                    print("failed to grab frame")
                    break
            
                # Get height and width of webcam frame
                height, width = frame.shape[:2]
            
                # Define ROI Box Dimensions
                top_left_x = int(width / 3)
                top_left_y = int((height / 2) + (height / 4))
                bottom_right_x = int((width / 3) * 2)
                bottom_right_y = int((height / 2) - (height / 4))
            
                # Draw rectangular window for our region of interest   
                cv2.rectangle(frame, (top_left_x, top_left_y), (bottom_right_x, bottom_right_y), 255, 3)
            
                # print(frame)#for testing only
                # Crop window of observation we defined above
                cropped = frame[bottom_right_y:top_left_y, top_left_x:bottom_right_x]
                # print(cropped)
                cv2.imshow('want recognize this product? ESC for yes, space fore next photo', cropped)
                cv2.imshow('Set object in blue square press SPACE if ok then press escape', frame)
            
                # Flip frame orientation horizontally we do that because 
                # it is easier for us to put the object in the front of the camera
                frame = cv2.flip(frame, 1)  # comment this later
                # cv2.imshow('flipped frame', frame)#comment this later
            
                k = cv2.waitKey(1)
                if k % 256 == 27:
                    # ESC pressed
                    print("Escape will closing a window and jump to next stage")
                    break
                elif k % 256 == 32:
                    # SPACE pressed
                    # img_name = "opencv_frame_{}.png".format(img_counter)
                    img_name = "maro_image/Object1.png".format(img_counter)
                    # cv2.imwrite(img_name, frame)
                    cv2.imwrite(img_name, cropped)
                    print("{} written!".format(img_name))
                    img_counter += 1
                    cv2.imshow('Press escape If you happy with this photo', cropped)
            
            cam.release()
            
            cv2.destroyAllWindows()

Functions used for object recognition.

Comparison with object number one.

Comparison with object number two.

Comparison with object number three.

Video Capturing

Main loop that recognize the objects and print information on the screen.

Matches are the objects that take an array data from the sift_detector and compare with the threshold. Threshold allows you to adjust sensitivity of the application. This value is set after tests.

NCI CHESS SOCIETY

In the first year of the course in 2020, I built a website for the chess club. The website was hosted on my private server and can be accessed at this
link. The website was built using, HTML, CSS, Java Script and Bootstrap.

SmartLuggage – The Application for Easy Packing

Setup Environment

Python code for capture the photo from the user.

Functions used for object recognition.

Video Capturing

NCI CHESS SOCIETY