projecs by owner
(自动玩微信小游戏跳一跳
中文说明请点这里
Requirements
- Python
- Opencv3
- Tensorflow
for Android
- Adb tools
- Android Phone
for IOS (Refer to this site for installation)
- iPhone
- Mac
- WebDriverAgent
- facebook-wda
- imobiledevice
Algorithms for Localization
- Multiscale search
- Fast search
- CNN-based coarse-to-fine model
For algorithm details, please go to https://zhuanlan.zhihu.com/p/32636329.
Notice: CV based fast-search only support Android for now
Run
Before running our code, connect to your phone via USB.
If Android phone, open the USB debugging at developer options enter adb devices to ensure that the list is not empty.
If iPhone, please ensure that you have a mac. Then following this link for preparation.
It is recommended to download the pre-trained model following the link below and run the following code
python nn_play.py --phone Android --sensitivity 2.045
You can also try play.py by running the following code
python play.py --phone Android --sensitivity 2.045
--phonehas two options: Android or IOS.--sensitivityis the constant parameter that controls the pressing time.nn_play.pyuses CNN-based coarse-to-fine model, supporting Android and IOS (more robust)play.pyuses multiscale search and fast search algorithms, supporting Android and IOS (it may fail sometimes in other phones)
Performance
Our method can correctly detect the positions of the man (green dot) and the destination (red dot).
It is easy to reach the state of art as long as you like.
But I choose to go die after 859 jumps for about 1.5 hours.
Demo Video
Here is a video demo. Excited!
Train Log & Data
CNN train log and train&validation data avaliable at
Training: download and untar data into any directory, and then modify self.data_dir in those files under cnn_coarse_to_fine/data_provider directory.
Inference: download and unzip train log dirs(train_logs_coarse and train_logs_fine) into resource directory.
How to Train CNN models by yourself?
- Download and untar data into any directory, and then modify
self.data_dirin those files undercnn_coarse_to_fine/data_providerdirectory. base.largeis model dir for coarse model,base.fineis model dir for fine model, other dirs undercnn_coarse_to_fine/configare models we don't use, but if you have interests, you can try train other models by yourself.- Run
python3 train.py -g 0to train your model,-gto specify GPU to use, if you don't have GPU, training model is not recommended because training speed with CPU is very slow. - After training, move or copy
.ckptfile to train log dirs(train_logs_coarseandtrain_logs_fine) for use.