Question about probability with infinities

Suppose I am randomly picking numbers. I can choose any number, so I have infinitely many possible choices each time. Obviously there are conceptual problems with randomly choosing a number from the entire number line, but let's assume I can do it. Is it possible to calculate the odds that I will pick a specific number given an infinite number of picks?

A different but related question. Suppose every day you give me 2 coins. Every day I throw the coins into a pile and randomly select one to give back to you. Obviously my pile of coins will increase by 1 coin every day. What are the odds of any given coin being present in the pile after an infinite number of days?

Does math exist to address these sorts of problems, or does the answer end up being "undefined"?

 

Log in or Sign up to hide all adverts.

The way that you "assume you can do it" matters.

You are in this territory: http://en.wikipedia.org/wiki/Axiom_of_choice

Depending on your assumptions (your second, "related" question involves a countable set - different from the "entire number line") I can see getting 0 as a limit, or simply not being able to define a reasonable "probability" in those circumstances.

 

Log in or Sign up to hide all adverts.

I'm pretty sure you do get zero as a limit, which is why I asked. It seems that at t=infinity there should be infinitely many coins in the pile, but if you work out the odds of any given coin being in the pile at infinity, it appears that the probability of it being there is zero.

 

Log in or Sign up to hide all adverts.

This can't be right. The probability of any particular coin being drawn probably falls off like 1/n, where n is the number of coins in the stack. The probability that any particular coin survives approaches 1---be sure to distinguish between "picked" and "not picked". So after the first day, a particular coin has a 50% probability of surviving. The next day, that coin has a 66% of surviving, etc.

The number line example is particularly easy. If we play pick a number with infinite digits, you'll have a zero probability of guessing the right number. But, of course, supposing you're game, the probability of you picking SOME number is 1. In fact, if you were to happen to guess to correct number, I would argue that my choice wasn't random at all---for example, I could have implicitly (out of laziness) chosen 10.0 because I can only remember three digits.

 

Agreed, for the first situation. I was sleeping, and mistook the description.

Not if he wants a technical, solid proof based on rigorous definitions.

 

The first problem IMHO is not well-defined at all: "Obviously there are conceptual problems with randomly choosing a number from the entire number line" is exactly right. "but let's assume I can do it" opens up the AC can of worms.

The second problem is a bit fuzzy: "What are the odds of any given coin being present in the pile after an infinite number of days?" Is the coin known to be in the pile at some time? (If not, I have to give it to you, and the source of my coins is not specified.) Can I give the coin back to you if I don't like it? (Yes or no changes the odds.)

So, "Suppose every day you give me 2 new coins. Every day I throw the coins into a pile and randomly select one to give back to you (the coin I return to you is not 'new'). Obviously my pile of coins will increase by 1 coin every day. What are the odds of any given coin present in the pile at some point remaining in the pile for an infinite number of days?" The answer to this question is zero.

 

You are correct about how the odds of a coin being drawn changes (decreases) every day as more coins are added to the pile. The odds of the coin still being in the pile after n days are (1-1/2)*(1-1/3)*(1-1/4)...(1-1/n). But if you take the limit of that series, you will get zero - which seems to imply that the odds of any coin still being in the pile at t=infinity is zero.

I agree with what you say here. But note that I was asking about my odds of guessing your number if I am allowed infinitely many guesses.

It makes sense that the odds of my picking the "correct" number each time should be exactly zero, but then again if the odds of me picking every number are zero, it seems that I would never be able to pick any. This is why I said there are conceptual problems with randomly picking a number on the number line.

 

You are quite right, I see that I didn't phrase that carefully enough. The way you rephrased it is the way that I intended for it to be interpreted. It seems pretty paradoxical that even though there should be infinitely many coins in the pile after infinity days, the odds of every coin that I have added to the pile being there is exactly zero.

That's why I said it was related to the first question. The odds of any given coin still being in the pile after infinite time seems to be zero, which would indicate that there are no coins in the pile. For every coin that I have ever added to the pile, the odds of it being there after infinity days are zero, so how could any of them be there? Yet clearly there should be infinitely many coins in the pile. Much like my first question, where I have infinitely many numbers to choose from, but the odds of choosing any given one of them is exactly zero.

Edit: There seem to be plenty of other ways to dress up this issue. For example, suppose every time I randomly pick a coin from the pile and give it back to you, you add it to your own pile of returned coins. After infinity days it seems that the odds of every coin that you have ever given me being returned to you are exactly one, which would imply that you have received all your coins back. But clearly you should only have received half your coins back.

 

You are misinterpreting/conflating/abusing the concepts of probability, limits, and cardinality here.

This second problem can be viewed from a finite perspective.

Suppose that on some day your pile has n[sub]0[/sub] coins in it. Before I give you my coins, I'm going to pick a coin from your pile, mark it in a way that is known to me but not to you, and put the coin back in your pile. Once per day from this point on, I'll give you two coins, which you will toss on your pile. You will then randomly pick a coin from your pile and give it to me. I'll toss that coin on a discard pile to keep the coins I give to you separate from the coins you give to me.

The probability that the specially marked coin has made it to the discard pile on the first day is 1/(n[sub]0[/sub]+2). Thus the probability that it remains in your pile after the first day's transactions is (n[sub]0[/sub]+1)/(n[sub]0[/sub]+2). The probability that it remains in your pile falls a bit every day; after N day that probability is (n[sub]0[/sub]+1)/(n[sub]0[/sub]+1+N). This obviously tends to zero as N grows unboundedly large, and thus the probability that this special coin is in my discard pile approaches 1.

This does not mean I have received every coin back. Read up on Cantor's Hotel.

 

The first problem can be resolved by introducing a continuous probability density on the real line. The probability you pick any given number is zero, but the probability that the picked number is in some interval may be nonzero. Even you pick countably infinitely many numbers at random, the odds that you pick a specific number is zero. You have to have uncountably many guesses to get positive odds.

In the second problem, the answer is zero as DH explained. Look up the Monty Hell Problem.

 

A better resolution of the first problem is to pick a distribution that does not introduce the reals and avoids the nonsense concept of a uniform distribution over the integers. For example, \(p(n) = \frac 1 3 2^{-|n|}\). This is obviously a probability mass function because \(\sum_{n=-\infty}^{\infty}p(n) = 1\).

The probability of not picking some number n on a single draw from some realizable mass distribution is \(1-p(n)\). The probability of never drawing this number in N draws is \((1- p(n))^N\), and thus the probability of drawing this number at least once in N draws is \(1-(1- p(n))^N\). Denote this as \(p(n;N)\).

Using the identity \(\frac{1-x^N}{1-x} = \sum_{n=0}^{N-1}x^n\) and setting \(x=1-p(n)\) yields

\(p(n;N) = 1-(1- p(n))^N = (1-(1-p(n)))\sum_{n=0}^{N-1}(1- p(n))^n = p(n)\sum_{n=0}^{N-1}(1- p(n))^n\)

In the limit \(N\to\infty\) this becomes \(p(n;\infty) = p(n)\sum_{n=0}^\infty(1- p(n))^n = p(n)\frac 1 {1-(1- p(n))} = \frac {p(n)}{p(n)} = 1\)

The probability of choosing a given number at least once in an infinite number of draws is one.

 

In fact we could make uncountably many guesses and still manage to get zero odds. For example if we chose some variant of the Cantor set.

 

The Cantor set is uncountably large. The OP is talking about choosing integers, a countable set. Try to come up with a probability mass function, not a probability distribution function, that leads to zero odds of picking a particular integer.

One example:

\(p(n) = \left{0 \qquad\qquad\quad\; \text{if}\;n<= 0 \\2^{-n}\quad\, \text{if}\;n>0\right.\)

The odds of drawing 0 (or any non-positive number) is zero. I don't think that is what the OP asked for.

 

That was the whole point! The set is uncountable but has measure zero.

My post is in reference to the quote I supplied. In addition, I see no mention of integers in the opening post.

 

Good point. However, since (a) the second part of the OP is about a discrete distribution, and (b) it is blatantly obvious that the probability of never choosing a given number from a distribution over an uncountably large set is one, I thought the OP was talking about a discrete distribution for part (a) of the OP.

 

Excellent post. I agree with Nassim Taleb. The only probability is that it's probable we can't predict the future.

 

I thought so.

Please Register or Log in to view the hidden image!

 

Thanks to all for the answers, especially D.H.

So apparently if I'm trying to guess integers, the odds of guessing the correct number after an infinite number of tries is one. What would be the odds of guessing the integer correctly if I only get 1 guess, or some finite number of guesses? I ask because it seems counter-intuitive that my odds of guessing correctly with one guess could be exactly zero, but that the odds increase to one with infinite guesses. Or am I misunderstanding something? Also, does it change anything if we're working with all real number instead of just integers?

 

That depends on how many items you have to guess from. If it is only one guess, then the probability of you getting it right is, naturally, 1/n

 

Thanks, and you're welcome.

Assuming you have a well-defined probability mass function, that is. A uniform distribution over all the integers is not a well-defined PMF.

You are implicitly assuming that you can define a uniform distribution over all the integers. An adroit mathematician could probably arrange to come up with any answer between zero and one as the result by changing the way the limits are computed.

Yes. Now you are moving from probability mass functions to probability distribution functions. If the cumulative distribution function is continuous at some particular value, the probability of drawing that value exactly is identically zero, as is the probability of drawing that number in a countably infinite number of drawings.