O'Reilly Radar

Four short links: 23 April 2013

April 23, 2013 - 3:54am

1. Drawscript — Processing for Illustrator. (via BERG London)
2. Archive Team Warrior — a virtual archiving appliance. You can run it to help with the ArchiveTeam archiving efforts. It will download sites and upload them to our archive. (via Ed Vielmetti)
3. Retro Vectors — royalty-free and free of charge.
4. TokutekDB Goes Open Source — a high-performance, transactional storage engine for MySQL and MariaDB. See the announcement.

Categories: Tech: General Technology

A different take on data skepticism

April 22, 2013 - 10:22am

Recently, the Mathbabe (aka Cathy O’Neil) vented some frustration about the pitfalls in applying even simple machine learning (ML) methods like k-nearest neighbors. As data science is democratized, she worries that naive practitioners will shoot themselves in the foot because these tools can offer very misleading results. Maybe data science is best left to the pros? Mike Loukides picked up this thread, calling for healthy skepticism in our approach to data and implicitly cautioning against a “cargo cult” approach in which data collection and analysis methods are blindly copied from previous efforts without sufficient attempts to understand their potential biases and shortcomings.

Well, arguing against greater understanding of the methods we apply is like arguing against motherhood and apple pie, and Cathy and Mike are spot on in their diagnoses of the current situation. And yet …

There is so much value to be gained if we can put the power of learning, inference, and prediction methods into the hands of more developers and domain experts. But how can we avoid the pitfalls that Cathy and Mike are rightly concerned about? If a seemingly simple method like k-nearest neighbors classification is dangerous in unskilled hands (and it certainly is), then what hope is there? Well, I would argue that all ML methods are not created equal with regard to their safety. In fact, it is exactly some of the simplest (and most widely used) methods that are the most dangerous.

Why? Because these methods have lots of hidden assumptions. Well, maybe the assumptions aren’t so much hidden as nodded-at-but-rarely-questioned. A good analogy might be jumping to the sentencing phase of a criminal trial without first assessing guilt: asking “What is the punishment that best fits this crime?” before asking “Did the defendant actually commit a crime? And if so, which one?” As another example of a simple-yet-dangerous method, k-means clustering assumes a value for k, the number of clusters, even though there may not be a “good” way to divide the data into this many buckets. Maybe seven buckets provides a much more natural explanation than four. Or maybe the data, as observed, is truly undifferentiated and any effort to split it up will result in arbitrary and misleading distinctions. Shouldn’t our methods ask these more fundamental questions as well?

So, which methods are better in this regard? In general, it’s those that explore model space in addition to model parameters. In the case of k-means, for example, this would mean learning the number k in addition to the cluster assignment for each data point. For k-nearest neighbors, we could learn the number of exemplars to use and also the distance metric that provides the best explanation for the data. This multi-level approach might sound advanced, and it is true that these implementations are more complex. But complexity of implementation needn’t correlate with “danger” (thanks in part to software engineering), and it’s certainly not a sufficient reason to dismiss more robust methods.

I find the database analogy useful here: developers with only a foggy notion of database implementation routinely benefit from the expertise of the programmers who do understand these systems — i.e., the “professionals.” How? Well, decades of experience — and lots of trial and error — have yielded good abstractions in this area. As a result, we can meaningfully talk about the database “layer” in our overall “stack.” Of course, these abstractions are leaky, like all others, and there are plenty of sharp edges remaining (and, some might argue, more being created every day with the explosion of NoSQL solutions). Nevertheless, my weekend-project webapp can store and query insane amounts of data — and I have no idea how to implement a B-tree.

For ML to have a similarly broad impact, I think the tools need to follow a similar path. We need to push ourselves away from the viewpoint that sees ML methods as a bag of tricks, with the right method chosen on a per-problem basis, success requiring a good deal of art, and evaluation mainly by artificial measures of accuracy at the expense of other considerations. Trustworthiness, robustness, and conservatism are just as important, and will have far more influence on the long-run impact of ML.

Will well-intentioned people still be able to lie to themselves? Sure, of course! Let alone the greedy or malicious actors that Cathy and Mike are also concerned about. But our tools should make the common cases easy and safe, and that’s not the reality today.

Related:

• Data skepticism

Categories: Tech: General Technology

Four short links: 22 April 2013

April 22, 2013 - 5:12am

1. Meshlab — open source, portable, and extensible system for the processing and editing of unstructured 3D triangular meshes.
2. HTML5 Video on iOS (Steve Souders) — While it’s true that Mobile Safari on iOS doesn’t buffer any video data as a result of the PRELOAD attribute, it does make other video requests that aren’t counted as “buffered” video. The number and size of the requests and responses depends on the video. For larger videos the total amount of data for these behind-the-scenes requests can be significant.
3. Space Monkey (Kickstarter) — distributed encrypted peer-to-peer cloud service using custom hardware. Not open source, which would make me nervous that I was buying a botnet client with storage capability. (via BERG London)
4. Matasano Crypto Challenges — Counting is not a hard problem. But cryptography is. There are just a few things you can screw up to get the size of a buffer wrong. There are tens, probably hundreds, of obscure little things you can do to take a cryptosystem that should be secure even against an adversary with more CPU cores than there are atoms in the solar system, and make it solveable with a Perl script and 15 seconds. Don’t take our word for it: do the challenges and you’ll see. People “know” this already, but they don’t really know it in their gut, and we think the reason for that is that very few people actually know how to implement the best-known attacks. So, mail us, and we’ll give you a tour of them.

Categories: Tech: General Technology

Four short links: 19 April 2013

April 19, 2013 - 3:46am

1. Bruce Sterling on Disruption — If more computation, and more networking, was going to make the world prosperous, we’d be living in a prosperous world. And we’re not. Obviously we’re living in a Depression. Slow first 25% but then it takes fire and burns with the heat of a thousand Sun Microsystems flaming out. You must read this now.
2. The Matasano Crypto Challenges (Maciej Ceglowski) — To my delight, though, I was able to get through the entire sequence. It took diligence, coffee, and a lot of graph paper, but the problems were tractable. And having completed them, I’ve become convinced that anyone whose job it is to run a production website should try them, particularly if you have no experience with application security. Since the challenges aren’t really documented anywhere, I wanted to describe what they’re like in the hopes of persuading busy people to take the plunge.
3. Tachyon — a fault tolerant distributed file system enabling reliable file sharing at memory-speed across cluster frameworks, such as Spark and MapReduce. Berkeley-licensed open source.
4. Jammit (GitHub) — an industrial strength asset packaging library for Rails, providing both the CSS and JavaScript concatenation and compression that you’d expect, as well as YUI Compressor, Closure Compiler, and UglifyJS compatibility, ahead-of-time gzipping, built-in JavaScript template support, and optional Data-URI / MHTML image and font embedding. (via Joseph Misiti)

Categories: Tech: General Technology

Sprinting toward the future of Jamaica

April 18, 2013 - 7:24am

Creating the conditions for startups to form is now a policy imperative for governments around the world, as Julian Jay Robinson, minister of state in Jamaica’s Ministry of Science, Technology, Energy and Mining, reminded the attendees at the “Developing the Caribbean” conference last week in Kingston, Jamaica.

Robinson said Jamaica is working on deploying wireless broadband access, securing networks and stimulating tech entrepreneurship around the island, a set of priorities that would have sounded of the moment in Washington, Paris, Hong Kong or Bangalore. He also described open access and open data as fundamental parts of democratic governance, explicitly aligning the release of public data with economic development and anti-corruption efforts. Robinson also pledged to help ensure that Jamaica’s open data efforts would be successful, offering a key ally within government to members of civil society.

The interest in adding technical ability and capacity around the Caribbean was sparked by other efforts around the world, particularly Kenya’s open government data efforts. That’s what led the organizers to invite Paul Kukubo to speak about Kenya’s experience, which Robinson noted might be more relevant to Jamaica than that of the global north.

Kukubo, the head of Kenya’s Information, Communication and Technology Board, was a key player in getting the country’s open data initiative off the ground and evangelizing it to developers in Nairobi. At the conference, Kukubo gave Jamaicans two key pieces of advice. First, open data efforts must be aligned with national priorities, from reducing corruption to improving digital services to economic development.

“You can’t do your open data initiative outside of what you’re trying to do for your country,” said Kukubo.

Second, political leadership is essential to success. In Kenya, the president was personally involved in open data, Kukubo said. Now that a new president has been officially elected, however, there are new questions about what happens next, particularly given that pickup in Kenya’s development community hasn’t been as dynamic as officials might have hoped. There’s also a significant issue on the demand-side of open data, with respect to the absence of a Freedom of Information Law in Kenya.

When I asked Kukubo about these issues, he said he expects a Freedom of Information law will be passed this year in Kenya. He also replied that the momentum on open data wasn’t just about the supply side.

“We feel that in the usage side, especially with respect to the developer ecosystem, we haven’t necessarily gotten as much traction from developers using data and interpreting cleverly as we might have wanted to have,” he said. “We’re putting putting more into that area.”

With respect to leadership, Kukubo pointed out that newly elected Kenyan President Uhuru Kenyatta drove open data release and policy when he was the minister of finance. Kukubo expects him to be very supportive of open data in office.

The development of open data in Jamaica, by way of contrast, has been driven by academia, said professor Maurice McNaughton, director of the Center of Excellence at the Mona School of Business at the University of the West Indies (UWI). The Caribbean Open Institute, for instance, has been working closely with Jamaica’s Rural Agriculture Development Authority (RADA). There are high hopes that releases of more data from RADA and other Jamaican institutions will improve Jamaica’s economy and the effectiveness of its government.

Open data could add $35 million annually to the Jamaican economy, said Damian Cox, director of the Access to Information Unit in the Office of the Prime Minister, citing a United Nations estimate. Cox also explicitly aligned open data with measuring progress toward Millennium Development Goals, positing that increasing the availability of data will enable the civil society, government agencies and the UN to more accurately assess success.

The development of (open) data-driven journalism

Developing the Caribbean focused on the demand side of open data as well, particularly the role of intermediaries in collecting, cleaning, fact checking, and presenting data, matched with necessary narrative and context. That kind of work is precisely what data-driven journalism does, which is why it was one of the major themes of the conference. I was invited to give an overview of data-driven journalism that connected some trends and highlighted the best work in the field.

I’ve written quite a bit about how data-driven journalism is making sense of the world elsewhere, with a report yet to come. What I found in Jamaica is that media there have long since begun experimenting in the field, from the investigative journalism at Panos Caribbean to the relatively recent launch of diGJamaica by the Gleaner Company.

diGJamaica is modeled upon the Jamaican Handbook and includes more than a million pages from The Gleaner newspaper, going back to 1834. The site publishes directories of public entities and public data, including visualizations. It charges for access to the archives.

Legends and legacies

Olympic champion Usain Bolt, photographed in his (fast) car at the UWI/Usain Bolt Track in Mona, Jamaica.

Normally, meeting the fastest man on earth would be the most memorable part of any trip. The moment that left the deepest impression from my journey to the Caribbean, however, came not from encountering Usain Bolt on a run but from within a seminar room on a university campus.

As a member of a panel of judges, I saw dozens of young people present after working for 30 hours at a hackathon at the University of the West Indies. While even the most mature of the working apps was still a prototype, the best of them were squarely focused on issues that affect real Jamaicans: scoring the risk of farmers that needed banking loans and collecting and sharing data about produce.

The winning team created a working mobile app that would enable government officials to collect data at farms. While none of the apps are likely to be adopted by the agricultural agency in its current form, or show up in the Google Play store this week, the experience the teams gained will help them in the future.

As I left the island, the perspective that I’d taken away from trips to Brazil, Moldova and Africa last year was further confirmed: technical talent and creativity can be found everywhere in the world, along with considerable passion to apply design thinking, data and mobile technology to improve the societies people live within. This is innovation that matters, not just clones of popular social networking apps — though the judges saw more than a couple of those ideas flow by as well.

In the years ahead, Jamaican developers will play an important role in media, commerce and government on the island. If attracting young people to engineering and teaching them to code is the long-term legacy of efforts like Developing the Caribbean, it will deserve its own thumbs up from Mr. Bolt. The track to that future looks wide open.

Disclosure: the cost of my travel to Jamaica was paid for by the organizers of the Developing the Caribbean conference.

Categories: Tech: General Technology

Four short links: 18 April 2013

April 18, 2013 - 6:20am

1. The Well Deserved Fortune of Satoshi Nakamoto — I can’t assure with 100% certainty that the all the black dots are owned by Satoshi, but almost all are owned by a single entity, and that entity began mining right from block 1, and with the same performance as the genesis block. It can be identified by constant slope segments that occasionally restart. Also this entity is the only entity that has shown complete trust in Bitcoin, since it hasn’t spend any coins (as last as the eye can see). I estimate at eyesight that Satoshi fortune is around 1M Bitcoins, or 100M USD at current exchange rate. Author’s credible. (via Hacker News)
2. Houdini (Github) — C library for escaping and unescaping UTF-8-encoded HTML, according to OWASP guidelines.
3. The $12 Gongkai Phone (Bunnie Huang) — gongkai isn’t a totally lawless free-for-all. It’s a network of ideas, spread peer-to-peer, with certain rules to enforce sharing and to prevent leeching. It’s very different from Western IP concepts, but I’m trying to have an open mind about it.
4. Jan Chipchase on Google Glass (All Things D) — Any idiot can collect data. The real issue is how to collect data in such a way that meets both moral and legal obligations and still delivers some form of value. An interesting observation, one of many within this overview of the usability and third-party user experience of Google Glass-like UIs.

Categories: Tech: General Technology

Four short links: 17 April 2013

April 17, 2013 - 6:49am

1. Computer Software Archive (Jason Scott) — The Internet Archive is the largest collection of historical software online in the world. Find me someone bigger. Through these terabytes (!) of software, the whole of the software landscape of the last 50 years is settling in. (And documentation and magazines and …). Wow.
2. 7 in 10 Doctors Have a Self-Tracking Patient — the most common ways of sharing data with a doctor, according to the physicians, were writing it out by hand or giving the doctor a paper printout. (via Richard MacManus)
3. opsmezzo — open-sourced provisioning tools from the Nodejitsu team. (via Nuno Job)
4. Hacking Secret Ciphers with Python — teaches complete beginners how to program in the Python programming language. The book features the source code to several ciphers and hacking programs for these ciphers. The programs include the Caesar cipher, transposition cipher, simple substitution cipher, multiplicative & affine ciphers, Vigenere cipher, and hacking programs for each of these ciphers. The final chapters cover the modern RSA cipher and public key cryptography.

Categories: Tech: General Technology

What is probabilistic programming?

April 16, 2013 - 9:03am

Probabilistic programming languages are in the spotlight. This is due to the announcement of a new DARPA program to support their fundamental research. But what is probabilistic programming? What can we expect from this research? Will this effort pay off? How long will it take?

A probabilistic programming language is a high-level language that makes it easy for a developer to define probability models and then “solve” these models automatically. These languages incorporate random events as primitives and their runtime environment handles inference. Now, it is a matter of programming that enables a clean separation between modeling and inference. This can vastly reduce the time and effort associated with implementing new models and understanding data. Just as high-level programming languages transformed developer productivity by abstracting away the details of the processor and memory architecture, probabilistic languages promise to free the developer from the complexities of high-performance probabilistic inference.

What does it mean to perform inference automatically? Let’s compare a probabilistic program to a classical simulation such as a climate model. A simulation is a computer program that takes some initial conditions such as historical temperatures, estimates of energy input from the sun, and so on, as an input. Then it uses the programmer’s assumptions about the interactions between these variables that are captured in equations and code to produce forecasts about the climate in the future. Simulations are characterized by the fact that they only run in one direction: forward, from causes to hypothesized effects.

A probabilistic program turns this around. Given a universe of possible interactions between different elements of the climate system and a collection of observed data, we could automatically learn which interactions are most effective in explaining the observations — even if these interactions are quite complex. How does this work? In a nutshell, the probabilistic language’s runtime environment runs the program both forward and backward. It runs forward from causes to effects (data) and backward from the data to the causes. Clever implementations will trade off between these directions to efficiently home in on the most likely explanations for the observations.

Better climate models are but one potential application of probabilistic programming. Other models include: shorter and more humane clinical trials with fewer unneeded side effects and more accurate outcomes; machine perception that transcends the capabilities of the now-ubiquitous quadcopters and even Google’s self-driving cars; and “nervous systems” that fuse data from massively distributed and noisy sensor networks to better understand both the natural world and artificial environments.

Of course, any technology this general carries a lot of uncertainty around its development path and eventual impact. So much depends on complex interactions with other technology threads and, ultimately, social factors and regulation. With all possible humility, here is one sample from the predictive distribution, conditioned on what we know so far:

• Phase I — Probabilistic programming will transform the practice of data science by unifying anecdotal reasoning with more reliable statistical approaches. If data science is first and foremost about telling stories, then probabilistic programming is in many ways the perfect tool. Practitioners will be able to leverage the persuasive power of narrative, while staying on firm quantitative ground.
• Phase II — Practitioners will really start to push the boundaries of modeling in fundmental ways in order to address many applications that don’t fit well into the current machine learning, text mining, or graph analysis paradigms. Many real-world datasets are a mixture of tabular, relational, textual, geospatial, audiovisual, and other data types. Probabilistic programs can weave all of these pieces together in natural ways. Current solutions that claim to integrate heterogeneous data typically do so by beating it all into a similar form, losing much of the underlying structure along the way.
• Phase III — Probabilistic programming will push well into territory that is universally recognized as artificial intelligence. As we’re often reminded, intelligent systems are very application-specific. Good chess algorithms are unlike Google’s self-driving car, which is totally different from IBM’s Watson. But probabilistic programs can be layered and modularized, with subsystems that specialize in particular problem domains, but embedded in a shared fabric that recognizes the current context and brings appropriate modeling subsystems to bear.

What will it take to make all this real? The conceptual underpinnings of probabilistic programming languages are well in hand, thanks to trailblazing work by research groups at MIT, UMass Amherst, Microsoft Research, Harvard, and elsewhere. The core challenge at this point is developing performant inference engines that can efficiently solve the very wide range of models that these languages can express. We’ll also need new debugging, optimization, and visualization tools to help developers get the most from these systems.

This story will take years to play out in full, but I expect we’ll see real progress over the next three to four years. I’m excited.

Want to learn more? BUGS is a probabilistic programming language originally developed by statisticians more than 20 years ago. While it has a number of limitations around expressivity and dataset size, it’s a great way to get your feet wet. Also check out Rob Zinkov’s tutorial post, which includes examples of several models. Church is the most ambitious probabilistic programming language. Don’t miss the tutorials, though it may not be the most accessible or practical option until the inference engine and toolset mature. For that reason, factorie might be a better bet in the short term, especially if you like Scala, or Microsoft Research’s infer.net with C# and F# bindings. The proceedings from a recent academic workshop provide a great snapshot of the field as of late 2012. Finally, this video from a long-defunct startup that I co-founded contains one stab at explaining many of the concepts underlying probabilistic programming referred to under the more general term probabilistic computing:

Categories: Tech: General Technology

Four short links: 16 April 2013

April 16, 2013 - 7:21am

1. Triage — iPhone app to quickly triage your email in your downtime. See also the backstory. Awesome UI.
2. Webcam Pulse Detector — I was wondering how long it would take someone to do the Eulerian video magnification in real code. Now I’m wondering how long it will take the patent-inspired takedown…
3. How Microsoft Quietly Built the City of the Future — The team now collects 500 million data transactions every 24 hours, and the smart buildings software presents engineers with prioritized lists of misbehaving equipment. Algorithms can balance out the cost of a fix in terms of money and energy being wasted with other factors such as how much impact fixing it will have on employees who work in that building. Because of that kind of analysis, a lower-cost problem in a research lab with critical operations may rank higher priority-wise than a higher-cost fix that directly affects few. Almost half of the issues the system identifies can be corrected in under a minute, Smith says.
4. UDOO (Kickstarter) — mini PC that could run either Android or Linux, with an Arduino-compatible board embedded. Like faster Raspberry Pi but with Arduino Due-compatible I/O.

Categories: Tech: General Technology

Four short links: 15 April 2013

April 15, 2013 - 4:05am

1. Know Your HTTP Posters (GitHub) — A0-posters about the HTTP protocol.
2. Crowdserfing — when a large corp uses crowd-sourced volunteering for its own financial gain, without giving back. It offends my sense of reciprocity as well, but nobody is coerced into using Google Maps or contributing data to it. How do we decide what is “right”?
3. Exposed Webcam Viewer — hotels in Russia, lobbies in California, and blinking lights in the darkness from all around the world. (via Hacker News)
4. Beauty and Joy of Computing — an introductory computer science curriculum developed at the University of California, Berkeley, intended for non-CS majors at the high school junior through undergraduate freshman level. Uses Snap, a web-based implementation of Scratch.

Categories: Tech: General Technology

Four short links: 12 April 2013

April 12, 2013 - 4:46pm

1. Wikileaks ProjectK Code (Github) — open-sourced map and graph modules behind the Wikileaks code serving Kissinger-era cables. (via Journalism++)
2. Plan Your Digital Afterlife With Inactive Account Manager — you can choose to have your data deleted — after three, six, nine or 12 months of inactivity. Or you can select trusted contacts to receive data from some or all of the following services: +1s; Blogger; Contacts and Circles; Drive; Gmail; Google+ Profiles, Pages and Streams; Picasa Web Albums; Google Voice and YouTube. Before our systems take any action, we’ll first warn you by sending a text message to your cellphone and email to the secondary address you’ve provided. (via Chris Heathcote)
3. Leo Caillard: Art Games — Caillard’s images show museum patrons interacting with priceless paintings the way someone might browse through slides in a personal iTunes library on a device like an iPhone or MacBook. Playful and thought-provoking. (via Beta Knowledge)
4. Lanyrd Pro — helping companies keep track of which events their engineers speak at, so they can avoid duplication and have maximum opportunity to promote it. First paid product from ETecher and Foo Simon Willison’s startup.

Categories: Tech: General Technology

Predictive analytics and data sharing raise civil liberties concerns

April 12, 2013 - 11:39am

Last winter, around the same time there was a huge row in Congress over the Cyber Intelligence Sharing and Protection Act (CISPA), U.S. Attorney General Holder quietly signed off on expanded rules on government data sharing. The rules allowed the National Counterterrorism Center (NCTC), housed within the Department of Homeland Security, to analyze the regulatory data collected during the business of government for patterns relevant to domestic terrorist threats.

Julia Angwin, who reported the story for the Wall Street Journal, highlighted the key tension: the rules allow the NCTC to “examine the government files of U.S. citizens for possible criminal behavior, even if there is no reason to suspect them.” 

On the one hand, this is a natural application of big data: search existing government records collected about citizens for suspicious patterns of behavior. The action can be justified for counter-terrorism purposes: there are advanced persistent threats. (When national security is invoked, privacy concerns are often deprecated.) The failure to "connect the dots" using existing data across government on Christmas Day 2009 (remember the so-called "underwear bomber?") added impetus to getting more data in the NCTC’s hands. It’s possible that the rules on data retention were extended five years because the agency didn’t have the capabilities it needed. Data mining existing records offers unprecedented opportunities to find and detect terrorism plots before they happen.

On the other hand, the changes at the NCTC that were authorized back in March 2012 represent a massive data grab with far-reaching consequences. The changes received little public discussion prior to the WSJ breaking the story, and they seem to substantially override the purpose of the Federal Privacy Act that Congress passed in 1974. Extension of the rules happened without public debate because of what effectively amounts to a legal loophole. Post proposed changes to the Federal Register, voila. Effectively, this looks like an end run around the Federal Privacy Act.

Here’s the rub: according to Angwin, DoJ Chief Privacy Officer Nancy Libin:

“… raised concerns about whether the guidelines could unfairly target innocent people, these people said. Some research suggests that, statistically speaking, there are too few terror attacks for predictive patterns to emerge. The risk, then, is that innocent behavior gets misunderstood — say, a man buying chemicals (for a child’s science fair) and a timer (for the sprinkler) sets off false alarms. An August government report indicates that, as of last year, NCTC wasn’t doing predictive pattern-matching.”

It’s hard to say whether predictive data analytics are now in use at NCTC. It would be surprising if there isn’t pressure to experiment, given the expansion of “predictive policing” in cities around the U.S.. There stand to be significant, long-lasting repercussions if the center builds capacity to apply that capability at large scale without great care and informed Congressional oversight.

One outcome is a dystopian scenario straight out of science fiction, from “thoughtcrime” to presumptions of guilt. Alistair Croll highlighted some of the associated issues involved with big data and civil rights last year.

As Angwin pointed out, the likelihood of a terrorist attack in the U.S. remains low as compared to other risks Americans face every day from traffic, bees or lifestyle decisions. After 9/11, however, public officials and Congress have had little risk tolerance. As a result, vast, expensive intelligence and surveillance infrastructure in the U.S. has been massively expanded, with limited oversight and very little accountability, as documented in “Top Secret America.”

When intelligence officials have gone public to the press as whistle-blowers regarding overspending, they have been prosecuted. Former National Security Agency staffer Thomas Drake spoke at the 2011 Web 2.0 Summit about his experience. We talked about it in a subsequent interview, below:

The new rules have been in place now for months, with little public comment upon the changes. (Even after it was relaunched, the nation doesn’t seem to be reading the Federal Register. These days, I’m not sure how many members of the DC media do, either.) I’m unsure whether it’s fair to blame the press, though I do wonder how media resources were allocated during the “horse race” of the presidential campaigns last year. Now, the public is left to hope that the government oversees itself effectively behind closed doors.

I would find a recent “commitment to privacy and civil liberties” by the Department of Homeland Security more convincing if the agency wasn’t confiscating and searching electronic devices at the border without a warrant. 

Does anyone think that the privacy officers whose objections were overruled in the internal debates will provide the effective counterweight protecting the Bill of Rights will require in the years to come?

Categories: Tech: General Technology

Data skepticism

April 11, 2013 - 9:00am

A couple of months ago, I wrote that “big data” is heading toward the trough of a hype curve as a result of oversized hype and promises. That’s certainly true. I see more expressions of skepticism about the value of data every day. Some of the skepticism is a reaction against the hype; a lot of it arises from ignorance, and it has the same smell as the rich history of science denial from the tobacco industry (and probably much earlier) onward.

But there’s another thread of data skepticism that’s profoundly important. On her MathBabe blog, Cathy O’Neil has written several articles about lying with data — about intentionally developing models that don’t work because it’s possible to make more money from a bad model than a good one. (If you remember Mel Brooks’ classic “The Producers,” it’s the same idea.) In a slightly different vein, Cathy argues that making machine learning simple for non-experts might not be in our best interests; it’s easy to start believing answers because the computer told you so, without understanding why those answers might not correspond with reality.

I had a similar conversation with David Reiley, an economist at Google, who is working on experimental design in social sciences. Heavily paraphrasing our conversation, he said that it was all too easy to think you have plenty of data, when in fact you have the wrong data, data that’s filled with biases that lead to misleading conclusions. As Reiley points out (pdf), “the population of people who sees a particular ad may be very different from the population who does not see an ad”; yet, many data-driven studies of advertising effectiveness don’t take this bias into account. The idea that there are limitations to data, even very big data, doesn’t contradict Google’s mantra that more data is better than smarter algorithms; it does mean that even when you have unlimited data, you have to be very careful about the conclusions you draw from that data. It is in conflict with the all-too-common idea that, if you have lots and lots of data, correlation is as good as causation.

Skepticism about data is normal, and it’s a good thing. If I had to give a one line definition of science, it might be something like “organized and methodical skepticism based on evidence.” So, if we really want to do data science, it has to be done by incorporating skepticism. And here’s the key: data scientists have to own that skepticism. Data scientists have to be the biggest skeptics. Data scientists have to be skeptical about models, they have to be skeptical about overfitting, and they have to be skeptical about whether we’re asking the right questions. They have to be skeptical about how data is collected, whether that data is unbiased, and whether that data — even if there’s an inconceivably large amount of it — is sufficient to give you a meaningful result.

Because the bottom line is: if we’re not skeptical about how we use and analyze data, who will be? That’s not a pretty thought.

Categories: Tech: General Technology

Four short links: 11 April 2013

April 11, 2013 - 6:56am

1. A General Technique for Automating NES Games — software that learns how to play NES games and plays them automatically, using an aesthetically pleasing technique. With video, research paper, and code.
2. rietveld — open source tool like Mondrian, Google’s code review tool. Developed by Guido van Rossum, who developed Mondrian. Still being actively developed. (via Nelson Minar)
3. KPI Dashboard for Early-Stage SaaS Startups — as Google Docs sheet. Nice.
4. Life Without Sleep — interesting critique of Provigil as performance-enhancing drug for information workers. It is very difficult to design a stimulant that offers focus without tunnelling – that is, without losing the ability to relate well to one’s wider environment and therefore make socially nuanced decisions. Irritability and impatience grate on team dynamics and social skills, but such nuances are usually missed in drug studies, where they are usually treated as unreliable self-reported data. These problems were largely ignored in the early enthusiasm for drug-based ways to reduce sleep. [...] Volunteers on the stimulant modafinil omitted these feedback requests, instead providing brusque, non-question instructions, such as: ‘Exit West at the roundabout, then turn left at the park.’ Their dialogues were shorter and they produced less accurate maps than control volunteers. What is more, modafinil causes an overestimation of one’s own performance: those individuals on modafinil not only performed worse, but were less likely to notice that they did. (via Dave Pell)

Categories: Tech: General Technology

Four short links: 10 April 2013

April 10, 2013 - 6:39am

1. HyperLapse — this won the Internet for April. Everyone else can go home. Check out this unbelievable video and source is available.
2. Housing Simulator — NZ’s largest city is consulting on its growth plan, and includes a simulator so you can decide where the growth to house the hundreds of thousands of predicted residents will come from. Reminds me of NPR’s Budget Hero. Notice that none of the levers control immigration or city taxes to make different cities attractive or unattractive. Growth is a given and you’re left trying to figure out which green fields to pave.
3. Converting To and From Google Map Tile Coordinates in PostGIS (Pete Warden) — Google Maps’ system of power-of-two tiles has become a defacto standard, widely used by all sorts of web mapping software. I’ve found it handy to use as a caching scheme for our data, but the PostGIS calls to use it were getting pretty messy, so I wrapped them up in a few functions. Code on github.
4. So You Want to Build A Connected Sensor Device? (Google Doc) — The purpose of this document is to provide an overview of infrastructure, options, and tradeoffs for the parts of the data ecosystem that deal with generating, storing, transmitting, and sharing data. In addition to providing an overview, the goal is to learn what the pain points are, so we can address them. This is a collaborative document drafted for the purpose of discussion and contribution at Sensored Meetup #10. (via Rachel Kalmar)

Categories: Tech: General Technology

The re-emergence of time-series

April 9, 2013 - 2:57pm

My first job after leaving academia was as a quant 1 for a hedge fund, where I performed (what are now referred to as) data science tasks on financial time-series. I primarily used techniques from probability & statistics, econometrics, and optimization, with occasional forays into machine-learning (clustering, classification, anomalies). More recently, I’ve been closely following the emergence of tools that target large time series and decided to highlight a few interesting bits.

Time-series and big data

Over the last six months I’ve been encountering more data scientists (outside of finance) who work with massive amounts of time-series data. The rise of unstructured data has been widely reported, the growing importance of time-series much less so. Sources include data from consumer devices (gesture recognition & user interface design), sensors (apps for “self-tracking”), machines (systems in data centers), and health care. In fact some research hospitals have troves of EEG and ECG readings that translate to time-series data collections with billions (even trillions) of points.

Search and machine-learning at scale

Before doing anything else, one has to be able to run queries at scale. Last year I wrote about a team of researchers at UC Riverside who took an existing search algorithm (dynamic time-warping 2) and got it to scale to time-series with trillions of points. There are many potential applications of their research, one I highlighted is from health care:

… a doctor who needs to search through EEG data (with hundreds of billions of points), for a “prototypical epileptic spike”, where the input query is a time-series snippet with thousands of points.

As the size of data grows, the UCR dynamic time-warping algorithm takes time to finish (it takes a few hours for time-series with trillions of points). In general (academic) researchers who’ve spent weeks or months collecting data are fine waiting a few hours for a pattern recognition algorithm to finish. But users who come from different backgrounds (e.g. web companies) may not be as patient. Fortunately “search” is an active research area and faster (distributed) pattern recognition systems will likely emerge soon.

Once you scale up search, other interesting problems can be tackled. The UCR team is using their dynamic time-warping algorithm in tasks like classification, clustering, and motif 3 discovery. Other teams are investigating techniques from signal-processing, pattern recognition, and trajectory tracking.

Some data management tools that target time-series

One of the more popular sessions at last year’s HBase Conference was on OpenTSDB, a distributed, time series database built on top of HBase. It’s used to store and serve time series metrics, and comes with tools (based on GNUPlot) for charting. Originally named OpenTSDB2, KairosDB was written primarily for Cassandra (but also works with HBase). OpenTSDB emphasizes tools for readying data for charts (interpolating to fill in missing values), KairosDB distinguishes between data and the presentation of data.

Startup TempoDB offers a reasonably priced, cloud-based service for storing, retrieving, and visualizing time-series data. Still a work in progress SciDB is an open source database project, designed specifically for data intensive science problems. The designers of the system plan to make time-series analysis easy to express within SciDB.

(1) I worked on trading strategies for derivatives, portfolio & risk management, and option pricing.

(2) From my earlier post: In a recent paper, the UCR team noted that “… after an exhaustive literature search of more than 800 papers, we are not aware of any distance measure that has been shown to outperform DTW by a statistically significant amount on reproducible experiments”.

(3) Motifs are similar subsequences of a long time series; shapelets are time series primitives that can be used to speed up automatic classification (by reducing the number of “features”).

This post was originally published on strata.oreilly.com.

Categories: Tech: General Technology

Privacy vs. speech

April 9, 2013 - 2:26pm

A week or so ago this link made its way through my tweet stream: “Privacy and the right to be forgotten.” Honestly I didn’t really even read it. I just retweeted it with a +1 or some other sign of approval because the notion that my flippant throwaway comments on the interwebs would be searchable forever has always left me a bit unsettled. Many times I’ve thought “Thank God the Internet wasn’t around when I was 20, because the things I would have said then online would have been order of magnitudes stupider than the stupidest things I say now.” I haven’t gotten any smarter, but I am a little bit better at filtering, and I rarely drink these days.

But today I read this piece from Stanford Law Review on the subject. And it’s smart. As is this simpler summary on NPR.

In so many domains the Internet creates these dichotomous tensions. There are two things we want and the Internet enables either, or neither, but not both.

I personally don’t think we need this kind of law. However, eventually it will become obvious that the cost of storing every damned thing I’ve ever uttered online exceeds any conceivable or achievable ROI from mining it. Hopefully, as companies realize this, they’ll offer a “feature” to solve this problem by letting me, and people like me, establish preferences for time to live and/or time to keep. For example, I’d be perfectly happy if Twitter enabled a one week time to live on every tweet I posted. They are meant to be ephemeral and it would be more than fine with me if their lifespans matched the level of thought I put into them.

Categories: Tech: General Technology

Four short links: 9 April 2013

April 9, 2013 - 6:50am

1. Automated Essay Grading To Come to EdX (NY Times) — shortly after we get software that writes stories for us, we get software to read them for us.
2. AMD Calls End of Moore’s Law in Ten Years (ComputerWorld) — story based on this video, where Michio Kaku lays out the timeline for Moore’s Law’s wind-down and the spin-up of new technology.
3. Addressing Human Trafficking Through Technology (danah boyd) — technologists love to make tech and then assert it’ll help people. Danah’s work on teens and now trafficking steers us to do what works, rather than what is showy or easiest.
4. Product Management (Rowan Simpson) — hand this to anyone who asks what product management actually is. Excellent explanation.

Categories: Tech: General Technology

An IPO by any other name

April 8, 2013 - 2:59pm

When Tableau goes public this summer, its shares will trade on NASDAQ under the apt ticker symbol “DATA.” Tickers are arguably less important now than they’ve ever been, since computers have removed much of the ambiguity they’re meant to resolve, but an interesting ticker symbol always stirs my fascination with corporate archaeology.

Some executives see prestige in quirky ticker symbols, or those that are just one letter long, and the exchanges take advantage of that to attract listings. It was rumored, for instance, that the New York Stock Exchange was reserving “M” and “I” for Microsoft and Intel, respectively, should they ever decide to ditch NASDAQ, and it finally gave up M in 2007 when Federated Department Stores re-listed itself as Macy’s. Since then, the exchanges have fallen over themselves to list tech companies, and NASDAQ and the New York Stock Exchange have handed over “Z” to Zillow and “P” to Pandora.

But the list of single-letter ticker symbols is strikingly Ozymandian. Two letters ahead of Zillow, and only about 43% larger by valuation, is United States Steel, the biggest corporation in the world at its founding by J.P. Morgan in 1901, but marginal enough 90 years later that it was removed from the Dow Jones Industrial Average.

At “Y” on the New York Stock Exchange is Alleghany Corp., today a smallish insurance firm, but at one time a swashbuckling holding company that controlled almost a fifth of the U.S. railway network, including the Chesapeake & Ohio and the New York Central, one of the country’s biggest, which it won in a dramatic proxy battle. It was a proto-conglomerate in the eventual mold of ITT and Gulf+Western, and in the middle of the 20th century it did as many railroad companies, and got the hell out of the railroad business. Its ancillary investments, after a few acquisitions and divestments, added up to an insurance business. (When, in 1970, Penn Central Transportation filed for the biggest bankruptcy to date, its parent company similarly survived and held an insurance business alongside some of the railroad’s real estate. Its successor company, American Premier Underwriters, continued to own New York’s Grand Central Terminal and the air rights above its tracks, leasing them to New York’s Metropolitan Transportation Authority, until 2006 when it sold them to a group of real-estate investors.)

I thought of closing this post by clarifying that I don’t wish the fate of U.S. Steel and Alleghany on Tableau and Zillow, but a century of existence for any company is a remarkable achievement, particularly for one that came about at a high point in investor enthusiasm for its industry.

Photo: Times Square – NASDAQ by luisvilla, on Flickr

Categories: Tech: General Technology

Four short links: 8 April 2013

April 8, 2013 - 6:28am

1. mozpay — a JavaScript API inspired by google.payments.inapp.buy() but modified for things like multiple payment providers and carrier billing. When a web app invokes navigator.mozPay() in Firefox OS, the device shows a secure window with a concise UI. After authenticating, the user can easily charge the payment to her mobile carrier bill or credit card. When completed, the app delivers the product. Repeat purchases are quick and easy.
2. Firefox Looks Like it Will Reject Third-Party Cookies (ComputerWorld) — kudos Mozilla! Now we’ll see whether such a cookie policy does deliver a better user experience. Can privacy coexist with a good user experience? Answers on a tweet, please, to @radar.
3. How We Lost the Web (Anil Dash) — excellent talk about the decreasing openness and vanishing shared culture of the web. See also David Weinberger’s transcription.
4. 3D From Space Shuttle Footage? — neat idea! Filming in 3D generally requires two cameras that are separated laterally, to create the parallax effected needed for stereoscopic vision. Fortunately, videos shot from Earth orbit can be converted to 3D without a second camera, because the camera is constantly in motion.

Categories: Tech: General Technology