Evidence, Induction and Drinking Games

The first example I thought of would either not be very interesting or get needlessly technical, so consider the following variant of the Experiment:

Suppose you are charged with the task of interrogating prisoners from two groups, called A and B--all members of A are truthful with probability 1/4 and members of B are truthful with probability 1/5. There are N prisoners in total, and you know that A of them are in group A.

You wish to separate the prisoners (you want to kill as many As and spare as many Bs as you can, say), so to begin you select one at random with no information at all about which group he or she belongs to.

You wish to test the hypothesis H: "The prisoner is a member of group A". Based on the previous paragraph, it would be sensible to take:

O(H,¬H) = A/(N-A)

ie

W(H,¬H) = log A - log(N-A)

To begin, you ask the question "Are you a member of group A?", to which the prisoner responds "Yes". The evidential statement E1 is "The prisoner responded 'Yes' when asked if he is an A".

Clearly, P(E1|H) = 1/4, since a 'Yes' answer would be the truth if the prisoner were an A. If not, the P(E1|¬H) = 4/5, since a 'Yes' response would be a lie. Thus

W(H,¬H|E1) = log 5 - 4 log 2

If the prisoner had answered 'no', the evidential statement would be ¬E1: "The prisoner responded 'No' when asked if he is an A". Now P(¬E1|H) = 3/4 and P(¬E1|¬H) = 1/5, so

W(H,¬H|¬E1) = log 5 + log 3 - 2 log 2

For generality's sake, suppose you ask this question a total of q times. The jth response is an evidential statement having the form of E1 or ¬E1 above, so if there where r 'yes' answers and s 'no' answers, we obtain, assuming answers to questions are independent (as required by the method):

S(H) = log A - log (N-A) + (r+s) log 5 + s log 3 - (4r+2s) log 2

Note that it would be redundant to ask the prisoner about their membership in group B (provided all prisoners are in one of the two groups) -- by this I mean, you may as well ask the original question once more for each time you'd like to ask this new question. How else might we revise S(H)? If this were an actual situation, there might be all manner of things to observe, but with the given information, the possible evidence is a lot more limited.

Whether it is prudent to make a decision with no further evidence depends on the actual value of S(H). First, if s is very large and r is comparatively small, S(H) will be significantly larger than the prior evidence, indicating that H is a sensible conclusion. This is intuitively correct, because if H is true, we are liable to get a lot of 'no's to the question, since A members most often lie. Similarly, if ¬H is true, it is unlikely we'll get many 'no's to the question, since members of B are both not members of A and usually liars. The exact reverse is true if r is large compared to s, and it matches our intuition for the same reason.

We have some kind of scale, given by the Experiment, for determining what values of S(H) can be considered 'large' or 'small' for the purposes of accepting or rejecting H (although I'd like to hear what people have to add to that). If S(H) is of sufficiently large magnitude, than to shake our faith in our conclusion, any new evidence must have a weight of very large magnitude, in the opposing direction.

In this case, we'd only worry about introducing additional evidence if r and s are relatively close in value.

@ bbarr: I don't see what problems there are in introducing and conditioning on evidence in this example. If you don't either, I will introduce progressively more 'subjective' examples until you do object.

Or, you could just make them Ro-Sham-Bo. Round robin, single elimination.

Originally posted by FreakyKBH
Or, you could just make them Ro-Sham-Bo. Round robin, single elimination.

Originally posted by DoctorScribbles
Your method has several flaws.

First, it is incoherent. How can a competition be both round robin and single elimination?

Second, your method is not guaranteed to terminate. The players can collude to always play rock, so that nobody ever loses.

Third, I don't see any connection between the method and jusification of belief. How would y ...[text shortened]... t the game?

I believe the method RC proposes is superior to yours on each of these metrics.

Originally posted by FreakyKBH

B's are idiots if they can't figure out how to cheat at the game, thus deserving of death.

Originally posted by royalchicken
That's circular; it's taken as axiomatic that the hypothetical Bayesian genocide monster want's to spare Bs and kill As. As might be Bayesians and Bs frequentists, or something; it doesn't matter because by assumption Bs do not deserve death.

Originally posted by FreakyKBH
O?