NSA data mining of phone records

[Stryder]

The mathematics is completely out on both counts as there would be far more variables applied.

"Data Mining" has been about for a number of years, most commonly you'll see it in day-to-day operations through the use of a search engine on the internet. The way a search engine works for the most part is first it has to collect a "view" of a website through the use of a "Robot" program. This information is then housed in a server as a cache which then has an "Agent" program iterate through line by line to compile information on what that cache content contains. Depending on how that Agent is written defines what information is looked for (through "filters") and how that information is that stored in a "fast indexing" format within a database. (The cache's can then be purged, although in recent years companies have a tendency to keep a cache to show what's available if something is down and also likely to show what content was there in the cases of litigation)

A "spider" is a program that attempts to follow all the links within the cached content and identify other pages for the "Robots" to be sent to, it doesn't deal with the pages content just the site structuring and source locations.

Then we actually get to the part about the actual "Search Engine", which itself attempts to pair "filters" (as provided by words separated by spaces, prefixed with -/+ or surrounded by comments) The fast indexing database is searched at that time to find the link to the cached page and of course URL of the actual page.
Additional algorithms can then be applied to find "similarities" in the number of instances of those particular words used in the filter through all the sites listings the agent's have been through.

The same can be said about any "Data Mining" operation, it's split down into procedural components to make it easy to manage and no algorithm alone would be capable of creating "Matches" without enough data being presented. (Imagine you want an algorithm to look for things out of place, how can you do so if you haven't got enough information to identify what "Normal" looks like? This is why the number of data records collected and used by the various governments.)

The concern people have isn't the fact they are being watched by an automated observer following predefined algorithmic patterns, the concern itself is who makes those patterns and why. After all it might be right in the current climate to look for pattern match involving a particular archetype of person (i.e. a would be terrorist), however If the powers that be wanted to hunt people down at a Eugenics level (Singling out the poor, the disabled or those with socio-dysfunctional family ties), it would be easy to change it to operate like that. (Which is reminscent of what the Nazi movement was all about.)

 

[ElectricFetus]

It is people behind those machines.

http://www.thenewspaper.com/news/16/1688.asp

Red light cameras? What that got to do with all our internet and phone activities being processed by data mining algorithms by the NSA to try to find terrorist?

The above talks about a lack of due process, innocent until proven guilty.

http://www.bostonglobe.com/metro/20...es-everyone/1qoAoFfgp31UnXZT2CsFSK/story.html

What the problem with this?

As far as you choosing to tell someone your information that is irrelevant to the issue of the government taking your personal data without permission nor probable cause.

National security is probable cause, most of this data is public, like your license plate number on your car. I'm I fine with regulations to prevent humans from accessing that information willly-nillly, but we have got to accept that machines will soon track our every move, inside and outside of our homes, nothing can be done to stop this.

It is a violation of the right to be secure in our persons papers effects without probable cause.

Well technically they had the legal right to the phone records, thanks to the patriot act.

 

[Captain Kremmen]

There is no other connection with Soviet oppression and genocide, except for the usual paranoia and hype.

I half agree.
No meaningful comparison can be made between US information gathering, and the bloody terror of Stalinist Russia, granted.
But desire to control intrusion is not simply paranoia.
Americans need to discuss what is private and what is public information.
You could extend your 4th amendment to cover it, if you allowed that private information is a possession.
privacy of the person and possessions as against unreasonable searches.
"Searches" fits quite well I think.
http://law2.umkc.edu/faculty/projects/ftrials/conlaw/rightofprivacy.html

 

[iceaura]

"geoff"] Grammar aside, this does not illustrate in what way the Democrats do not engage in precisely the same kind of vicious politicking and partisanship that the Republicans do. I see no difference - especially given the events of the last few weeks - because there is no difference.

This thread is not the time, but one obvious place for you to look at the differences between the Party tactics would be in the filibustering stats, another would be in the media rhetoric and one-Party mass market "news" outlets, another would be in the phenomenon of "primarying" disloyal Congressmen, another would be in the promulgation of slanders untrue by Party officials in Party venues and public occasions, another would be in the denial and disavowal of one's own political promises and efforts in alignment with Partisan goals (to the point of individual Reps speaking and voting against bills they themselves authored), another would be the constant barrage of finger-wagging juvenile discourtesy and "you lie!" ranting personal attack on the sitting President from Party officials and Party representatives, another would be the headlong Party dive into the fantasy dis-fest of dishonest Superpac and corporate-backed media efforts (James O'Keefe getting repeated play after exposure, Breitbart dying one step ahead of the lawsuit, etc) and so forth and so on.

There is no parallel behavior among "the Democrats" as a group, and never has been. That's not a compliment - the Dems have been wimps and wonks in a really dirty street fight, and we have lost as a country accordingly, but there it is. And there has been no particular wavecrest of vicious politicking and partisanship by anyone in the last few weeks - it's been business mostly as usual, with a small increase in factual basis for the Republican rhetorical slandering (some of it is actually accurate, recently, and not outright lies, which is unusual) the major change - that would be a reduction in viciousness, not an increase, mostly by chance.

From what bizarre corner of irrelevancy did the name "Obama" come from, there?
Oh, didn't you know? He was elected president of the United States a few years back.

Oh, that corner. Completely irrelevant, but the name is on TV a lot so not so bizarre maybe - one can sort of see where your mind dredged it from. Look - I'm sure you had some point to make, but Obama has little to do with the "good guy/bad guy" observation you were ostensibly replying to. Can we just keep everything sort of organized, one thing at a time?

The data mining exposure is of a program launched illegally by W&Cheney, patched up and justified post hoc by a Patriot Act and amendments that quite a few Dems and essentially no Reps voted against, continued by an authoritarian center-right Administration with the usual excuses of security and benign intentions, and now presented as if it were familiar and inevitable, like the weather, something we can do nothing about, a choice we have no alternative to.

If we had impeached the President who launched it, as several analysts noted the first President ever to admit in public to an impeachable offense, we would not be facing its "inevitability". But we dug ourselves in, and now we have to find a way to dig out of this nasty little pit.

 

[iceaura]

But the in the military world I would date the US capability to no later than the mid-70s,

The US government has had the capability since the first telephone was installed. It has had the capability of reading everyone's mail since the Post Office was invented.

When it started to be employed for clandestine domestic monitoring full scale, wholesale, without even the pretense of probable cause or a warrant, and eventually openly defended, was in the early 2000s.

This societal permission can be revoked.

 

[Aqueous Id]

I half agree.
No meaningful comparison can be made between US information gathering, and the bloody terror of Stalinist Russia, granted.

Hi CK. Yes the hyperbole is out of place here.

But desire to control intrusion is not simply paranoia.

From the police and the military?

Americans need to discuss what is private and what is public information.

Am I correct in recalling that you are in Australia? Yes, I do think this discussion has been going on here in the US ever since the computerization of medical and banking information became systematized by operations like EDS (Ross Perot) and TRW (who are connected to a US plot to assassinate the liberal candidate in Australia in the 70s, BTW).

You could extend your 4th amendment to cover it, if you allowed that private information is a possession.

Or the 5th Amend. (no person shall be deprived of life liberty & property w/o due process of law.) The issue there is "due process of law". If we give this power to the government, it's hard to complain about them using it.

privacy of the person and possessions as against unreasonable searches. "Searches" fits quite well I think.

That's even weaker than the 5th because it's waived under "probable cause". The most bizarre case of this is a woman who ended up in jail when she protested the police response to her refusal to wear a seat belt, and part of her complaint (the seizure of her car, her personal property and of her person) constituted a 4th Amend. violation. Unfortunately she lost in the higher courts, which dealt a severe blow to civil rights vs police intrusion. They ruled that there was "probable cause" since not wearing a seat belt was a motor vehicle violation, and regardless how minor the offense, "probable cause" makes no distinction about the offense severity.

I don't think the civil rights provisions in the laws of any country will ever trump the powers granted to the (usu. executive) branch which does surveillance to enforce laws and provide for the common defense. Nor do I think that even if the legislative bodies of those countries mounted a reform movement would the public ever be willing to downgrade their police and military, out of fear of degrading the public safety. It might happen as a movement, but I suspect that even if laws were passed and constitutions were amended, it would probably mostly amount to lip service, while the covert agencies and police continued with business as usual, feeling empowered to do as they deem necessary by the common perception of public will. Right or wrong loses its meaning in this context, because the executive making the decision to intrude begins to operate from a higher ("more supreme") law - the will of the people . . . subject to individual interpretation, of course, and nearly impossible to prosecute under abuse of power, since the defense and police secrecy laws necessarily shield them. You can't ask a judge to enforce a "lower" law, even if it's the actual supreme governing law that should prevent such abuses, without evidence that a violation occurred. And you can't get evidence when it's protected by police and military secrecy laws. All of this would have to be revamped, which is probably an Gordian knot of infinite complexity. But the other side of this is that abuse of private information is already made illegal by the identity theft laws. So we all we have to ask is: do we or do we not expect our governments to obey the laws? Otherwise, if we expect them to violate them, then it seems that no legal reforms would have any effect. They would presumably then just ignore the new laws, as needed to do their work.

In part this is what motivates insiders to leak state secrets, unless of course they're strictly motivated by money. The fellow who leaked the info about the SIGINT operation in Alice Springs in the 70s (Christopher Boyd - and his cohort) only wanted to sell the info so the Ruskies would take notice of him, believing that he was an insider motivated by money, so they would take the data and exploit it. He knew that exposing the operation would preclude the need for an assassination.

His true motivation was strictly political and idealistic - that the assassination plot on the Australian candidate was a heinous moral offense only legally justified by the covert agencies as a technicality of law, and motivated strictly as a way to protect the US base in Australia. (The liberals were probably going to shut it down.) Boyd didn't care about what the Russians might be conspiring against the US and the cost or risk associated with shutting down the surveillance operation. He strictly reacted to the likelihood of a US assassination operation taking place over there, and the moral indignation that it aroused in him.

I brought this up since we have so many folks from the Eastern Tropic of Capricorn here (some of the best contributors, I might add), and since it's a story that has significance to the issue of the day, even though it was so long ago. As I think you and others already noted, personal information is more freely given to private companies under the premise that they could not or would not abuse it, at least not with tanks and helicopters (or drones, if you're really paranoid) to come at you. To me that logic - as applied in the OP - is ludicrous, since once the information is given to a private concern, it's much more easily leaked than if it were in police or military custody. And of course it assumes some kind of doomsday scenario. I realize there's no consolation in this, it just seems to me that there is a sort of double bind here. We either want a Big Brother to watch our back, or we don't want him to have eyes at all.

Therein lies the rub.

 

[milkweed]

Yes, the closest thing we have in the US to concentration camps are prisons. Other than that, what's the connection?

A SURVEILLANCE program the soviets could have only dreamed of.

Your the one introducing gulags

The only tangible threat to US public safety has been the violence perpetrated by maniacs.

No, not really. This is the mass hysteria I spoke of earlier.

Reality:
http://www.washingtonpost.com/blogs/wonkblog/files/2013/04/terrorist-attacks-since-1970.png

More reality:
http://www.washingtonpost.com/blogs/wonkblog/files/2013/04/groups.png

http://www.washingtonpost.com/blogs...t-facts-about-terrorism-in-the-united-states/

How do YOU explain the dramatic reduction in terrorist attacks in the usa between 1970 and 2000? Before 9/11. Before the 'patriot act'. Before PRISM/NSA phone record data mining?

Since we can assume that the US intelligence community always has, and always will, collect domestic intel to wage war and prevent attacks, who would ever doubt their exploitation of every bit of information they might find useful in that regard?

lol we dont have to assume. And exploit in what way?

What kind of an interview was it?

An irregular one. There was no lawyer present. It was not recorded. By the time Todashev was shot, he had apparently been interrogated by three agents for five hours. And then? Who knows?

At first he drew the knife while being interviewed. Then he acquired it during a break from the interview. Then it ceased to be a knife and became a sword, then a pipe, then a metal pole, then a broomstick, then a table, then a chair. In one account all the agents were in the room at the time of the attack; in another, all but one had mysteriously departed, leaving the remaining officer to face his assailant alone.

http://www.guardian.co.uk/commentisfree/2013/jun/03/ibragim-todashev-drones-policy-obama

Too bad they didnt interview this guy BEFORE the boston bombing eh?

The precogs sit in a room which is perpetually in half-darkness, constantly talking nonsense to themselves that is incoherent until it is analyzed by a computer and converted into predictions of the future.

http://en.wikipedia.org/wiki/The_Minority_Report

After all... computer models are nothing more that mechanized psychics gathering their insight from the ethernet rather than the aether of old. And nobody lies or tells tall tales on the ethernet do they? LOL

I am glad my job doesnt depend on terrorist's existence.

 

[milkweed]

Snip

Still waiting for
your name
Address
phone number
last years income.

Put your money where your mouth is. You the one who doesnt care about such things. Doesnt think they are private. If its good enough for the government to harvest without your permission why not me?

You think I am going to exploit it?

You want me to swear to uphold the constitution? Will that pacify your fears?

But as far as your attempt to justify the illegal data harvesting, plenty of bad laws are overturned. Like the red light cameras.

 

[ElectricFetus]

Still waiting for

ARE YOU A MACHINE??? I've told plenty of machines all that information repeatedly! I'm fine with machines going through that information for say finding terrorist, if and or when that information actually hits human eyes, that, THAT is when I have a problem! When I said "thanks to the patriot act" I was being sarcastic: that act is a travesty for personal freedoms. Do you understand, do you understand what I'm saying or in the event you are a machine are your semantics and language processing algorithms programed for this kind of sophisticated discerning?

 

[milkweed]

blah blah blah

There are PEOPLE running those machines.

In 2004, police tracked Canadian reporter Kerry Diotte via automated license scans after he wrote articles critical of the local traffic division. A senior officer admitted to inappropriately searching for the reporter’s vehicle in a license scan database in an attempt to catch Diotte driving drunk.

http://www.bostonglobe.com/metro/20...es-everyone/1qoAoFfgp31UnXZT2CsFSK/story.html

A bit closer to home:

http://www.myfoxtwincities.com/stor...inappropriately-accessed-drivers-license-data

Of course someone with clearance is going to exploit the info. IIRC the DNR situation came to light via someone asking to see how many times their own data had been accessed (possibly the police officer in the second part) . 4 years this guy was off the clock voyering personal protected data.

 

[Tiassa]

Someone Check the Trains

This is a give and take. It would not surprise me, for instance, if the FBI has a file on me. I've been arrested twice for minor, common criminal offenses. I subscribed to a Communist newspaper in college; have used mobile phones, and even pagers back in the day, to arrange drug deals; advocated Anarchism and a gaggle of other controversial political, religious, and social ideas.

At one point, it turned out there was a warrant for my ass for nine years. Part of the reason I didn't know was that there was an occasion when an officer should have been obliged to arrest me, and said nothing. He did mention my license suspension, which was news to me because I was oblivious and stupid, but I certainly wasn't arrested as I should have been.

My point being, in all these years they've never come for me.

I get the whole ethical shock and horror thing, but if the reality was as bad as our fears are supposed to be, I would have "been disappeared" a long, long time ago.

Repeal the AUMF, and at least tweak the Patriot Act. That will solve the current context of the problem.

And, of course, they'll keep doing it. They'll do it until we literally tack them to the wall for doing it.

And, yes, we as a society screw up by this process, frequently. So far, it's well enough with the people to not get out the hammers and nails.

But there's that whole thing about liberty and vigilance, too. This might be a particularly pointed manifestation of the issue, but I sincerely doubt we're about to call the whole thing off over it. What makes the current context significant is that it arises in an atmosphere of paranoid scandalmongering.

I'm actually surprised to hear the program's advocates speaking up. They certainly are helped in doing so by right-wing whiners like Sensenbrenner, whose present outrage is nothing more than insisting that he is and always has been an idiot.

And, yes, I wasn't expecting Franken to throw in on NSA's side, but he's trying to stake a sensible position and, just maybe, pick up the credibility to be the one who gets to say, "I'll tell you when to panic."

So far, everything sounds about on schedule, a rare thing, indeed, for the people's business.

 

[Captain Kremmen]

When the armed automatic drones start buzzing around America,
that's when you'll need to worry.
They'll make Robocop look like a wishy washy Liberal.

 

[Aqueous Id]

Yes, the closest thing we have in the US to concentration camps are prisons. Other than that, what's the connection?

A SURVEILLANCE program the soviets could have only dreamed of.
Your the one introducing gulags

You want to equate the use of intelligence gathering by two governments with different agendas. So you tacitly introduced gulags and whole lot more.

The only tangible threat to US public safety has been the violence perpetrated by maniacs.

No, not really. This is the mass hysteria I spoke of earlier.

The operative term is tangible. There has been no tangible threat on US public safety the scope of 9/11 since 9/11. As for mass hysteria, it's the predictable and intentional result of terrorism, so no surprise there. But the fears you are expressing are better characterized as the basis for mass hysteria since they are based on wild assumptions about things that you can't possibly know, since the information is obviously classified.

And exploit in what way?

The discussion revolves around wiretaps. I expanded that to include SIGINT (signals intelligence) activities of a broader scope, to include all surveillance technology that the US has at its disposal for collection on any person of interest, foreign or domestic. I think you may have underscoped the actual US capability a bit.

How do YOU explain the dramatic reduction in terrorist attacks in the usa between 1970 and 2000? Before 9/11. Before the 'patriot act'. Before PRISM/NSA phone record data mining?

What you really mean is, all was quiet on the home front until 9/11. I see no other logic that follows from this.

Since we can assume that the US intelligence community always has, and always will, collect domestic intel to wage war and prevent attacks, who would ever doubt their exploitation of every bit of information they might find useful in that regard?

lol we dont have to assume.

You are assuming a lot. You have no idea what goes on in NSA briefings nor do you have any clue about the character of the people making the decisions given to them by the consent of the governed. You're just assuming they are up to no good. So, yes, you do have to assume a lot to jump to conclusions.

And exploit in what way?

At present we are discussing wiretaps. I expanded that to include the broader scope of SIGINT.

What kind of an interview was it? An irregular one. There was no lawyer present. It was not recorded. By the time Todashev was shot, he had apparently been interrogated by three agents for five hours. And then? Who knows?

At first he drew the knife while being interviewed. Then he acquired it during a break from the interview. Then it ceased to be a knife and became a sword, then a pipe, then a metal pole, then a broomstick, then a table, then a chair. In one account all the agents were in the room at the time of the attack; in another, all but one had mysteriously departed, leaving the remaining officer to face his assailant alone.

What does this have to do with wiretapping/SIGINT/data mining?

Too bad they didnt interview this guy BEFORE the boston bombing eh?

That's precisely the logic for stepping up domestic surveillance.

The precogs sit in a room which is perpetually in half-darkness, constantly talking nonsense to themselves that is incoherent until it is analyzed by a computer and converted into predictions of the future.
http://en.wikipedia.org/wiki/The_Minority_Report

Great sci-fi. But not much more as far as this conversation goes.

After all... computer models are nothing more that mechanized psychics gathering their insight from the ethernet rather than the aether of old. And nobody lies or tells tall tales on the ethernet do they? LOL

By far most computer models are as far from the sci-fi psycho stereotype you seem to have in mind. They do mostly boring laborious tasks that, when humans performed them, probably contributed more to mental illness than you're giving credit for. Surveillance is much more specialized, and also mostly boring, except during actual conflict or engagement with a known bad guy. It's like searching for a needle in a haystack. No one cares about your personal business as much as you think, except for the identity thieves. They're the actual enemy, as far as we know.

I am glad my job doesnt depend on terrorist's existence.

In all likelihood a lot more than your job is riding on the success of police and military operations. That includes all the covert stuff we'll never know, first to intervene against nukes, and now, against the random psychopaths who like fiery explosions in crowded places.

 

[Aqueous Id]

When the armed automatic drones start buzzing around America,
that's when you'll need to worry.
They'll make Robocop look like a wishy washy Liberal.

Or they'll just be delivering aid to folks inundated by floods, etc. Of course that's the utterly boring scenario.

 

[Captain Kremmen]

This type of activity has been going on for decades, I guess the media every so often likes to cast a light on the matter to scare everyone. The media doesn't tell you that this kind of information gathering has been done since the government started the FBI, then the CIA then the NSA.

It's the scope and the level of intrusion that are different.
These people are ignoring the constitution, because they don't feel bound by it.
They think that the Government is above the law.
The secrecy was not for security, it was to avoid putting the activity under judicial control.

@Aqueous
Yes, the technology would have benefits too.
We shouldn't blame the drones.

 

[ElectricFetus]

There are PEOPLE running those machines.

People running a machine does not mean they see the data.

http://www.bostonglobe.com/metro/20...es-everyone/1qoAoFfgp31UnXZT2CsFSK/story.html

Question: what does this have to do with NSA data mining?

http://www.myfoxtwincities.com/stor...inappropriately-accessed-drivers-license-data

Seriously, what does this have to do with NSA data mining?

Of course someone with clearance is going to exploit the info. IIRC the DNR situation came to light via someone asking to see how many times their own data had been accessed (possibly the police officer in the second part) . 4 years this guy was off the clock voyering personal protected data.

and they will hopefully be caught and imprisoned, look we give our information redly online all the time, when you give out your credit card item for goods and services you hope that someone on the other end does not charge you for stuff you did not order, when you do your taxes you hope that the guy that processes it does not take you SSN and other information and identity thieve you, generally they don't, that just the risk we got to take. Tons and tons of phone records looked over my NSA mainframes is not going to translate into much if harassment by the government on the common man.

 

[Captain Kremmen]

With proper checks and controls, this activity could be made acceptable.
They want to check and control themselves. Not acceptable.
Snowden has done well to bring this into the open.

 

[milkweed]

In a rare public ruling by the nation’s most secretive judicial body, the Foreign Intelligence Surveillance Court ruled Wednesday that it did not object to the release of a classified 86-page opinion concluding that some of the U.S. government’s surveillance activities were unconstitutional.

http://openchannel.nbcnews.com/_new...to-release-of-opinion-on-illegal-surveillance

And from the comments section:

The secret court, decided to release its secret opinion, in which it is their opinion that the United States Government is/was violating the Constitution, in Secret.

You can rest easy now knowing that they took secret measures, to make sure their not violating our rights in secret, not that we will know what those measures are, or if they are even being enforced, because there secret.

 

[GeoffP]

When the armed automatic drones start buzzing around America,
that's when you'll need to worry.
They'll make Robocop look like a wishy washy Liberal.

That they'll never do.

As fools as they are, drones are a weapon for abroad.

 

[Bells]

Except that the public didn't know they were doing it. That's not transparency.

Selective memory? People have forgotten about the Patriot Act?

From Glenn Greenwald:



So, still illegal, apparently.

Seems Greenwald also forgot about the Patriot Act, which made it legal.

And for some reason, until now, Dems were very much against that Act. Until now.

Oh really?

It's a sad fact that American partisan politics do a level of damage to the body politic out of all proportion to the importance or intelligence of their proponents.

No one will disagree that this form of intelligence gathering is wrong because it is so intrusive. However, this big brouhaha seems hyped up. Everyone knew the Government was doing it. They had been doing it since the Patriot Act came into existence, if not from before that.