I can't stop thinking about algorithms. I am obsessed and I want to tell you why.
Let's be clear: I am not a data scientist. I am a guy who finds technology and applications of technology fascinating. I am not writing this for technology nerds. I am writing this for professionals that want a working knowledge of technology.
If you are reading this, then you understand computers. A computer is nothing more than rules programmed by a human. Those rules are then executed and create an output.
But algorithms are so much more; they are breathtaking. An algorithm is a computer writing it's own rules and then creating output from those rules.
It's easy to focus on the scary part of algorithms. In the Avengers movie, a super algorithm results in a machine -- Ultron -- bent on destroying the world. I will leave those scenarios to the Elon Musk's of the world.
Algorithms can do so much good
Think about any repetitive task you do. An algorithm can be created to solve that same task. Some algorithms are used for fun. For example, Facebook uses algorithms to suggest friends for you to connect with. Google Photos uses an algorithm to identify faces and group pictures of the same person together (which can lead to terrible results).
How Algorithms Work (and why they are Awesome)
Again, I am not a data scientist, but here is my simple explanation of how most if not all algorithms are created.
1. Create a seed set.
First you identify a seed set, which is the core learning that is taught to the algorithm. Yes, that's right, even a computer algorithm has to be taught something from a human! For example, with the Facebook algorithm, I'm almost certain that the algorithm was first fed a giant spreadsheet that contained information about individuals and how they were connected (you do know your data created Facebook, Google, and every other big data company you can think of, right?).
2. Feed the seed set to the algorithm.
The algorithm then reads all of the information it is fed and starts making its own rules. For example, the Facebook algorithm may determine: "Oh, I see, Jimmy likes Teenage Mutant Ninja Turtles, and he is connected with Bobby from the same city and Bobby also likes Teenage Mutant Ninja Turtles. I bet Jimmy also knows Steve from the same city with a love for Donatello. They should connect."
3. Human reviews the algorithm output.
A human (see, you are still needed!) then reviews the output of the application of the algorithm rules. In the Facebook example, a human might determine if Jimmy and Steve should actually connect on Facebook. Maybe they are part of rival gangs and the algorithm didn't recognize this. The human would then add this data to the spreadsheet and feed it back to the algorithm.
4. The algorithm rules are improved based on new input.
The algorithm creates new rules to account for the new information. "Don't connect rival gang members even if they live in the same city and like the Teenage Mutant Ninja Turtles."
5. Steps three and four continue indefinitely.
Now stop for a second and think about all the rules that are built up in your head about people you connect with. Maybe you prefer to hang out with people that brew beer or read Harry Potter. There are literally hundreds of millions of personal preferences that human beings use to associate with people.
What if you could store all of those preferences and use them to connect people?
Algorithms are Good for Insurance Workers
Now think about your work and all the stuff you know and all of the stuff your colleagues know about. What if all of that information could be fed into an algorithm and used to create rules. You could then use those rules to more quickly do your work.
I can hear you thinking "But then I will be out of a job." Therein lies the rub, one that has been discussed ad nauseum (over 9,000,000 results for a Google Search on "technology will destroy jobs"). Fatalists argue that algorithms and the advanced software programs they create, will destroy jobs. Famous technologist and investor Marc Andreesen expressed as much when he proclaimed in 2011 that "software is eating the world."
But what happens if software starts doing repetitive tasks previously done by humans? I believe humans find new ways to be productive. And, I believe history supports my theory. But that's a blog post for another day.
I will leave you with two questions.
What repetitive tasks do you despise?
Wouldn't it be great if you could offload these tasks to a computer?