Hi, I want to do a little project that works like “Video Sudoku Solver”

Hi, I want to do a little project that works like “Video Sudoku Solver”. I plan to use a webcam to watch a Chinese chess( similar to international chess) or chess , Chinese chessman is round with a different Chinese charater on center. (The international chess pieces is different 3 demention object.)

Then record every steps of the whole game. What I plan to do is: at the begin of a game, every chess pieces is in its initial position, so it is easy to detect chess board, and the original chess piece at expected position . After the game begin, I need to find when a plays hand move chess pieces, and follow its move detect the new positon. I guess rocord Chinese chess is simpler than international chess. I hope if after detect the circle, then every chess can be find, then the quesetion is to detect every chinese character in the circle. Because at the beginning, we would find the orignial picture of each pieces, so it seems not hard to find each pieces. Now the question is how to detect plays hand move in and move out? another question is the rotation of chess pieces.Because player usually put each pieces with a little rotation, so how to detect it? It seems hard to record international chess(to detect in 3 demention) If you have any ideas, please not hesitate to tell me! Thank u. This can be expand to a Apple App, or some other little applition in your mobile phone.if some good Artificial Intelligence(not as good as deep blue, but over average standand ) added to give a best advice for each move.

Answer 1

[Image Analyst]

I don't know what these pieces look like but I imagine many of them look identical. So to detect which piece moved you're going to have to take snapshots in between every move and then compare the latest snapshot against the prior one to determine which piece moved. You can probably just subtract the two images and then threshold and use bwareaopen to get rid of small noise patches and keep just the largest difference. If you know what pieces were located where at the start then you don't need to figure out what all the pieces are in each image because only one piece can move at a turn. I don't see any need to find every piece in every image, handle piece rotations, read Chinese characters, or use artificial intelligence - you only need to know initial positions and which piece moved from one turn to the next, which is easily determined.

[john]

Yes,only record the game, maybe it is simple.

So to detect which piece moved you're going to have to take snapshots in between every move and then compare the latest snapshot against the prior one to determine which piece moved

How to detect hand move in and move out????

But if expand this to a iphone App, i.e, sometimes, when I get trouble in a chess game, I take out iphone, and pretend to call someone, and take a picture of the chess board, and find a good solution. Usually each picture is taken at a little different position, so complex algorithms is to guarantee in this situation. Or in some final phase game.(In China, to play final phase chess, the game is almost to end, but in this stage, there is only one right move to win)

[Image Analyst]

I'm sure you don't want to try anything too ambitious for your first project so keep it simple. Maybe have the users just push a button when they're done to have the camera take a photo.

[john]

This is not a good idea. I think simple rocord solution is: when the hand is move in, then difference of the picture with the last one is over some threshold , when the hand move out if the threshold is under the threshold but over another small threshold, the move is detected. If the difference is also under the threshold, the player just keeping think and does not move this time. The webcam should be fixed. Do you think this can work?

[john]

I have a initial solve for this project.
First we can judge if the video is stable. When a hand is moving, the video is unstable.
this can be done by |Frame(n+1)-Frame(n)|+|Frame(n)-Frame(n-1)|>threshold. (C1)
if 3 sucessive
video has little difference, this means the video is stable.
Then the whole alrorithms can be:
(1)
initiale 
state=stable
input video, store it
(2)
detect if the video is stable according C1.
If the video state is change from stable to unstable then again back to stable
then maybe a move exists.
(3)detect if  a move exists, record the stable video.From two successional stable video (or snapshot) to detect if a move exists.
I am not good at stateflow, does anyone give me some advice to achieve above algorithms using stateflow ,simulink,d image processing and image acquireing toolbox without write any code?