28 Apr 11
Compare the moves actually made against an engine - for example I have already had one game where virtually every one of my opponent's moves was the same as Fritz 11. That makes it pretty clear I was playing an engine.
I am not sure the probabilities but I think I read once that if you have around 60% (?) of moves the same as a computer engine first preference then it is pretty strong evidence.
28 Apr 11
Originally posted by KailenDepends on the number of reasonable options for each move, and the length of the game of course.
Compare the moves actually made against an engine - for example I have already had one game where virtually every one of my opponent's moves was the same as Fritz 11. That makes it pretty clear I was playing an engine.
I am not sure the probabilities but I think I read once that if you have around 60% (?) of moves the same as a computer engine first preference then it is pretty strong evidence.
But charted over dozens of games, strong computer match-up becomes really damning.
28 Apr 11
Originally posted by Kailencan someone help me work out how you get a percentage.
Compare the moves actually made against an engine - for example I have already had one game where virtually every one of my opponent's moves was the same as Fritz 11. That makes it pretty clear I was playing an engine.
I am not sure the probabilities but I think I read once that if you have around 60% (?) of moves the same as a computer engine first preference then it is pretty strong evidence.
28 Apr 11
3 edits
Arena 3.0 can perform an automatic analysis of a game, and it tells you the number of matching moves. For example, 6 of 9 matching moves. It only does auto analysis of a single principal variation, though. If you want to do an analysis using multi-pv (matchup rate of top 3 choices, etc), you have to do it manually. (Arena will perform an auto-analysis with an engine set to multi-pv, but the results will be garbage.)
Does anyone know of some software that can do an auto analysis using multi-pv?
Edit: An answer to my own question - I just found something called ChessAnalyze, available at Chess.com. Claims to give analysis with top 3 choices using any UCI engine. Haven't tried it yet.(Looks like this is a demo only. Rats.)
29 Apr 11
2 edits
Originally posted by MontyMoosethankyou so much, now sorry to sound so dumb, but then again maybe its true, how
matches with the computer / total moves x 100 = % match with computer
14 matches in 20 moves:
14/20 = .7 x 100 = 70% match rate
2 matches in 20 moves:
2/20 = .1 x 100 = 10 % match rate
would this work for different engine choices, from the top three, for example, say
there was 15 overall engine matches out of a possible 20 non data base moves, 10 of
them were the first choice, 4 of them the second choice and 1 of them the third choice.
29 Apr 11
Originally posted by HabeascorpBlack's first-choice matchup rate for last 10 moves was 100% with Houdini 1.5a at 20 sec per move.
I am curious now. game ID6219092 if anyone feels like telling me the match up?
I'm not strong enough to judge whether a 2100 RHP rated would be capable of these moves, but the matchup rate does raise eyebrows.
(White's matchup rates for last 10 moves were:
40% of 1st choice
50% of top 2 choices
80% of top 3 choices)
🙂
29 Apr 11
Originally posted by robbie carrobieIn this case, it would be:
thankyou so much, now sorry to sound so dumb, but then again maybe its true, how
would this work for different engine choices, from the top three, for example, say
there was 15 overall engine matches out of a possible 20 non data base moves, 10 of
them were the first choice, 4 of them the second choice and 1 of them the third choice.
Best move - no.1 choice: 50%
Best move - no. 2 choice: 20%
Best move - no. 3 choice: 5%
Best move - up to no. 1 choice: 50%
Best move - up to no. 2 choice: 70%
Best move - up to no. 3 choice: 75%
- Joined
- 10 Apr 03
- Moves
- 48786
29 Apr 11
Computer analysis is making an impact on opening theory. This trend will only accelerate. Books that incorporate computer analysis are becoming commonplace. Sites such as the Open Encyclopedia of Chess openings allow players to choose their opening moves based upon the pre-analyzed computer output which is posted to the site.
The Open Encyclopedia of Chess Openings is a place where you can post computer analysis of opening positions, add commentary, post counter-analysis refuting someone else's analysis, build upon existing analysis that you agree with, etc.
It is a wiki-based, user-generated encyclopedia of chess openings. Each wiki page is a unique opening position. Pages may include computer analysis of the position, history, written explanations of the themes or motifs for the variation, etc.
The site attracts players who are interested in "advanced" chess (centaur chess). Chess players can also share their post-mortem analysis of correspondence or OTC games.
29 Apr 11
Originally posted by robbie carrobiefirst you discard all database moves, then analyse the rest for top 3 matchup against any decent engine. repeat that for a bunch of games (against strong opponents), tally up the results, compare to fischer's results. then ponder whether it's possible or not for an amateur to consistently outplay fischer.
can someone help me work out how you get a percentage.
that's it in a nutshell.
30 Apr 11
Originally posted by wormwoodHappy to see database / opening moves excluded from the match-up percentage, but shouldn't forced moves (and forcing moves) also be excluded? Unless I am horribly mistaken I imagine most sequences of forced moves (on both sides) would be near to 100% match-up with any engine. In my opinion it is the eyebrow raising "what-the-?" moves that show evidence of engine use, rather than a sequence of forced moves or the checkmate move.
first you discard all database moves, then analyse the rest for top 3 matchup against any decent engine. repeat that for a bunch of games (against strong opponents), tally up the results, compare to fischer's results. then ponder whether it's possible or not for an amateur to consistently outplay fischer.
that's it in a nutshell.
I also reckon that standards would/should change with time. As players interact with computers more often nowadays (e.g. today vs 10 years ago and 10 years ago vs 20 years ago) that the standards should also shift. So the match up percentage nowadays would be expected to be higher vs say 10 (or 20) years ago, even if you aren't using an engine to make your moves. For instance, in practicing a line/playing against a computer would it not be common to remember/follow sequences you first saw versus a computer player that you then use in a game? Statistically the answer would have to be yes. Is that cheating? I don't think remembering a sequence versus a computer is cheating, but it would probably have a very high % match up rate.
Unless there is some transparency to the process and a proven statistical methodology* to the tests then I remain a bit of a skeptic - I'm not saying we throw the baby out with the bathwater by any definition - I simply fear the risk of a false positive being issued against someone that isn't cheating.
*When I refer to a statistical methodology I'm talking about statistical confidence levels, sample sizes, sample selection etc. Something that you might study in a statistics paper rather than "I tested 20 games and got a 70% match up". To me that says nothing about the games that weren't tested, nor the confidence level, what moves were included/excluded, or the expected error rate etc. I'm no expert in this area but not seeing these sorts of things in posts causes me some mild concern.
Just my 2c
Andrew
30 Apr 11
Originally posted by andrew93The method is a crude method that I think is capable of identifying blatant cheaters, but no more than that. If someone is obtaining significantly high matchups over a big enough sample, then I do regard that as being very suspicious.
I simply fear the risk of a false positive being issued against someone that isn't cheating.
But borderline cases are indeed more debatable, especially when we consider how the borderline was derived. The bottom line is that we don't have hard figures for how well a modern GM can play CC today without engine assistance. And even if we had that, as you mention, maybe other factors could still influence the matchup. This doesn't make the method completely redundant, but it does complicate where the borderline is.
30 Apr 11
Originally posted by andrew93Happy to see database / opening moves excluded from the match-up percentage, but shouldn't forced moves (and forcing moves) also be excluded?
No, because who decides what is forced and what is not. Is a GM good enough? FIDE 2200? no-one under 1800? Once someone makes qualitative judgements about individual moves the "sample" is being manipulated.
Match up analysis measures just that. How many times the moves played match the first, second and third choice of software.
In my opinion it is the eyebrow raising "what-the-?" moves that show evidence of engine use, rather than a sequence of forced moves or the checkmate move.
This infers to me that you would prefer to ban on the basis of one move. Computers do not find moves that are not there to be found. There is no set of moves available to a computer that is not available to be made by any player of whatever ilk. Greenpawn34 has provided this forum with enough jaw dropping moves from time before the invention of computers to render your argument doubtful and again you are prejudging the sample.
So the match up percentage nowadays would be expected to be higher vs say 10 (or 20) years ago, even if you aren't using an engine to make your moves.
I think there is an argument that says match up rates should actually drop. As computers have now opened a 400 elo point gap or more over the best human players it may well mean that humans being measured against a higher standard of accuracy may well further short than hitherto. Thus whilst Capablanca achieved the best match up of the champions of yesteryear in one survey I read about a few years ago, might he now be lower what he is measured against calls into doubt moves that were previously thought to be accurate.
When I refer to a statistical methodology I'm talking about statistical confidence levels, sample sizes, sample selection etc. Something that you might study in a statistics paper
In every case I have seen posted by the likes of Squelchbelch or Zygalski on this forum, Processor speed, databases details and engine have been clearly stated.
Every time I read a post like yours I worry for the integrity and the future of the human vs human game. Every time myths, guesswork and apparently logical trains of thought are used to muddy the waters there is the gap in which the computer users can operate, to break the rules of sites like this which prohibit consultation of chess engines during games. Every book of games collections I have ever read will often show players career results in them. Amazingly all of our human chess heroes and heroines lose games and yet in the on line world there are players who barefacedly will recite the kinds of things you have posted here in an attempt to justify their unlikely levels of accuracy.
Sadly for the opening poster there is no way to tell if you are playing a computer user during a game. You can only do it afterwards with match up analysis and perhaps some other diagnostics. Even then you still have to ponder, as wormwood has said, how likely it is for an amateur to outplay Fischer.
In my more cynical moments I believe the writers of chess engines make most sales to players to "cheat" at online chess. The debates I read on Forums about whether one 3 litre computer engine is needed versus a 3.2 litre engine 😛 when all the players are claiming to do is allow the engine to analyse their latest blitz game does raise a disbelieving smile.
All you need is the ceiling for what match up rate is "impossible." Then you need the human willing to be tested in laboratory conditions to attempt to raise it on behalf of all the on line cheats. At present on line seems quite busy, but the lab sadly empty.
30 Apr 11
Originally posted by andrew93that's all been done. to death, I might add. some of the issues you bring up (as dozens or hundreds have before) are non issues, most are testable with control groups (the current masters differ from old ones much less than you'd think, statistically), and the math you can always crunch through yourself when in doubt.
Happy to see database / opening moves excluded from the match-up percentage, but shouldn't forced moves (and forcing moves) also be excluded? Unless I am horribly mistaken I imagine most sequences of forced moves (on both sides) would be near to 100% match-up with any engine. In my opinion it is the eyebrow raising "what-the-?" moves that show evidence of ...[text shortened]... ng these sorts of things in posts causes me some mild concern.
Just my 2c
Andrew
couple of practical points: firstly, no pre-computers era master has failed the test, neither otb master or cc. the people banned here have surpassed those match-up levels by an unnecessary large margin. secondly, the statistics of modern players improve only marginally over old ones. thirdly, it's been tested that including forced sequences makes only a tiny impact on the result. much, much, less than the normal tolerance levels for banning. same for 'memorizing computer lines' which gets killed by combinatorial explosion.
there are a billion threads like this if you use the search, and even though most of the actual figures and measures used (by rhp) are kept undisclosed, an abundance of clarifications about how it all works. (in the case someone doesn't do it all over here. I won't bother.)