Dark Mode

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

supertigim/elevator_buttons_recognition

Repository files navigation

Introduction

This project is to show how to detect and recognize buttons in an elevator for robotics.

  • Button Detection: tensorflow(<2.0) detection API
  • Button Recognition: OCR
  • Button Status (On/Off): Mean color value of each button

Environment

$ conda create -n detection python=3.7 pyqt=5
$ conda activate detection
(detection)$ git clone https://github.com/supertigim/elevator_buttons_recognition.g it
(detection)$ cd elevator_buttons_recognition
(detection)elevator_buttons_recognition$ pip install -r requirements.txt
(detection)elevator_buttons_recognition$ mkdir addons && cd addons
(detection)elevator_buttons_recognition/addons$ git clone https://github.com/tzutalin/labelImg.git
(detection)elevator_buttons_recognition/addons$ cd labelImg
(detection)elevator_buttons_recognition/addons/labelImg$ pip install -r requirements/requirements-linux-python3.txt
(detection)elevator_buttons_recognition/addons/labelImg$ cd ../..
(detection)elevator_buttons_recognition$ git clone https://github.com/tensorflow/models.git

When labelImge doesn't work properly,

(detection)elevator_buttons_recognition/addons/labelImg$ sudo apt-get install pyqt5-dev-tools
(detection)elevator_buttons_recognition/addons/labelImg$ sudo pip3 install -r requirements/requirements-linux-python3.txt
(detection)elevator_buttons_recognition/addons/labelImg$ make qt5py3
pyrcc5 -o libs/resources.py resources.qrc

Training

Before training, environment needs to be setup.

(detection)elevator_buttons_recognition$ sudo apt-get install protobuf-compiler
(detection)elevator_buttons_recognition$ cd models/research

# Once done, Don't need to do again
(detection)elevator_buttons_recognition/models/research$ wget -O protobuf.zip https://github.com/google/protobuf/releases/download/v3.0.0/protoc-3.0.0-linux-x86_64.zip
(detection)elevator_buttons_recognition/models/research$ unzip protobuf.zip
(detection)elevator_buttons_recognition/models/research$ protoc object_detection/protos/*.proto --python_out=.

# For each terminal or put it in .bashrc for convenience
(detection)elevator_buttons_recognition/models/research$ export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim

There are 5 steps with the additional step for monitoring

# 1. Create xmls with labelImg
(detection)elevator_buttons_recognition$ python ./addons/labelImg/labelImg.py

# 2. Convert xml to csv
(detection)elevator_buttons_recognition$ python xml_to_csv.py -i ./images/train/ -o ./annotations/train_labels.csv
(detection)elevator_buttons_recognition$ python xml_to_csv.py -i ./images/test/ -o ./annotations/test_labels.csv

# 3. Convert .csv to .record
(detection)elevator_buttons_recognition$ python generate_tfrecord.py --csv_input=./annotations/train_labes.csv --output_path=./annotations/train.record --img_path=images/train/
(detection)elevator_buttons_recognition$ python generate_tfrecord.py --csv_input=./annotations/test_labes.csv --output_path=./annotations/test.record --img_path=images/test/

# 4. Start training
(detection)elevator_buttons_recognition$ python train.py --logtostderr --train_dir=training/ --pipeline_config_path=training/ssd_inception_v2_coco.config

# (Optional) For visualization
(detection)elevator_buttons_recognition$ tensorboard --logdir=training

# 5. Conversion to .pb file
(detection)elevator_buttons_recognition$ python freeze_model.py --input_type image_tensor --pipeline_config_path ./training/ssd_inception_v2_coco.config --trained_checkpoint_prefix ./training/model.ckpt-200000 --output_directory ./frozen_model

Inference

(detection)elevator_buttons_recognition$ python main.py -m cam # Camera Streaming
# or
(detection)elevator_buttons_recognition$ python main.py -m image # Images Files
# or
(detection)elevator_buttons_recognition$ python main.py -m video # Video File

The pressed button on the image is recognized in red.

To-do List

  • Press Button Detection Improvement
  • Tensorflow 2.0 Implementation using Model Garden

Reference

How to build my own button detector

Papers in regard to elevator buttons detection

ETC

About

No description, website, or topics provided.

Resources

Readme

Stars

Watchers

Forks

Releases

No releases published

Packages

Contributors

Languages