Among other things

Military Is Awash in Data From Drones
By Christopher Drew

This is an interesting article on the state of the war on data. Laying the perfect foundation for a TFA moment of fairly massive proportions the devices we are creating to collect data are so good at doing it that we can't keep up with the raw material that they've created. An analogy - stuff everything you have into your washing machine, really cram it in there, be sure to add plenty of soap, then turn it on and go get a mop.

More after the jump...

This is what the process of iterative data collection might look like...
1. collect raw data in the field > 2. store data > 3. organize data > 4. analyze data > 5. sort data into discrete actionable groups || 1a. with each group treat data as raw and repeat at step 1 - integrating new data as you go. (soak-wash-rinse-repeat)

One solution could be to devise even more computational solutions to analyze said data. This is a problem because I think that people will be tasked with building a "fail-proof" system to do this analysis (which will inevitably fail - a la Things Fall Apart). This is a complicated process that requires careful consideration at each stage. One wrong decision could cause you to effectively "lose" important information. Perhaps they should devise better data collection protocols that has a better chance of retrieving usable data. Maybe there are two approaches to making the collection/analysis system better.

1. Put resources into building a smarter collection mechanism. So when data is collected some analysis is done on the spot - before it is catalogued with other data.
2. Put resources into building a smarter analysis mechanism. So when data is collected there are abundant, redundant layers of analysis ready to retrieve, organize and cross-reference it.

This is doubly interesting to me because an amazing professor, named Ugo Gagliardi, told me a story once about the earliest hard drives (when magnetic discs where becoming fast and small enough to mass manufacture). As he told it there was a kind of crisis in the industry early on. On one hand hard drive manufacturers could spend a lot of money trying to make a VERY smart hard drive controller. This very smart controller would effectively remember where everything was. On the other hand, a much cheaper approach would be to make the hard drives dumb - very very dumb. They spun, they wrote, they read - but the operating system would need to tell these drives what to write and where. The drives we use today are descendants of this early decision and, as you might imagine, the cheaper solution was the chosen. The result was a market with more competition where each member could compete on price to put their product in a computer with a range of operating systems.

*If* I have the problem correctly assessed - I wonder what a good solution is? It's not simple. On the supply side though (meaning the people and places being surveilled) it could get easier to overstuff the machine - the more brittle the system gets the less actual garbage it takes to undermine it. (GI/GO)

I wonder which will be chosen? Smart drones that can analyze on the fly? Fleshware using an army of smart analysts light the WW1 era rangefinders teams? Maybe - smart software, that will be called skynet it will be able to analyze and even direct new missions without having to slow down for our mushy emotional analysis... Eventually it will learn to command drones without us - integrating data and selecting new targets - realizing that the human race is illogical and therefore a threat to successful data collection and analysis (its sole goal). --- The skynet solution seems best :-O