As promised earlier this year, we at Vidya are proud to officially announce our newest course Analytics with Apache Spark. Spark is a cool technology making an enormous—and growing—impact in the Big Data space, so naturally there are a lot of courses out there. Ours is different.

Naturally we spend a lot of time on Spark itself with numerous code examples and challenging exercises, but we also stress the importance of things that have always mattered and still matter—architecture, security, and software engineering concepts like unit and integration testing, continuous integration, and continuous delivery. Courses seem to ignore these concerns even though they are actually more important in a complex, distributed environment.

We even survey many of the other leading analytics tools out there so you can get a feel for your options and make informed technical decisions.

All of this distinguishes our Spark course from all the others. We hope you will consider it if you want to make your mark in the Big Data landscape.

Feel free to check out our other courses too—Software Engineering in Java and Agile Software Project Management with Scrum.

I spent most of my career building software for U.S. government clients as a contractor, and one thing I noticed is just how bad the government is at running software projects. Take every bad thing every commercial software project ever did in the 1990’s, and it was built into the government software development process. It’s like modeling every high school after Bayside High.

Eventually, everyone else caught on with the infamous rollout of HealthCare.gov, but even then pundits and politicians only sought to examine that debacle through the prism of the politics of the Affordable Care Act. It frustrated me that people used HealthCare.gov as a proxy for their views on the law rather than a cautionary tale on how not to run IT projects.

Politics and ideology aside, the government has a technology problem.

Vidya is proud to be working with Thomson Reuters Special Services, a leading provider of threat detection solutions. Their software analyzes billions of public and proprietary records with innovative technology to deliver realtime, actionable intelligence to support sound decision making.

We have joined a team of senior engineers with a wide variety of expertise. Currently, we are using Play Framework in Scala as the web application framework with an AngularJS interface to a backend MongoDB database. Performance is critical, so aside from aggressive indexing, we are utilizing Akka to build a highly concurrent, resilient architecture. Aside from the current challenges, we look forward to helping evolve this application by re-architecting it into Nathan Marz’s Lambda Architecture with Hadoop and Elasticsearch.

We are lucky to be working with TRSS so we can learn from such a talented team and contribute our own expertise to such interesting, challenging work.

If you are in the Washington, DC, area and love free pizza, I will be speaking at Polyglot Programming DC a week from today on Scala, a hot programming language for everything from web development to “Big Data” to Android development.

Also, free pizza.

Every week Petr Zapletal, a software engineer at Cake Solutions, publishes a must-read post for Scala developers called quite intuitively This Week in #Scala. We are proud to announce that our latest tutorial Nine Reasons to Try Scala was featured in this week’s edition.

We would like to thank Petr for thinking enough of our tutorial to include it in This Week in #Scala. Please check it out this week and every week for the latest news on the Scala ecosystem.

Please take a look at my latest column for Government Computing News where I discuss why Kanban might be an easier agile alternative for government IT projects having trouble with the rigor of Scrum. Just to give you an idea, here is the unedited introduction.

Over the last decade, many have written about what agile software development offers to government IT. Their success has led to a GAO report, for which I was a contributor, on making agile work in the federal government and the U.S. Digital Services Playbook, which I wrote about for GCN.


Yet while government IT has improved, it has a long way to go. We witnessed the spectacular failure of the initial rollout of HealthCare.gov, and many far less visible failures happen all the time. One reason is that not enough government IT projects are agile. Another is that Scrum, the most popular agile framework, is hard.


In fact Scrum is so hard that the primary duty of the ScrumMaster is to serve as its guardian. Too often senior management pays lip service but does not make the commitment necessary for Scrum to succeed; even if it does, too often the team lacks the discipline to see it through.


Although I am certified in Scrum and teach a Scrum course, I am not so dogmatic as to ignore how hard it is to execute effectively. Thankfully, it isn’t our only hope. As Yoda once said, “There is another.”


Let’s examine why Kanban may in many cases be the better choice for government IT projects than Scrum.

You can read the rest of the column here and offer your own thoughts here.

In the applications I typically develop, I only care when strings are meaningful or when they aren’t because they are null (or nil, None, or Null depending on the language). Empty strings and strings containing only whitespace are this weird, in-between third category I really would prefer not to deal with. When I go to McDonald’s, I want to focus on the burgers and fries. I want absolutely nothing to do with the Filet-O-Fish.

Scala’s Option gives us a neat way to avoid meaningless strings if we wish.

Please take a look at my latest column for Government Computing News where I drop references to Brangelina and Kimye to explain why DevOps has become so popular in business and why it needs to be a thing in government as well. Just to give you an idea, here is the unedited introduction.

Terms like service-oriented architecture and big data have long been part of the pantheon of buzzwords that have captivated us while eluding any kind of real understanding. Recently joining them is DevOps. In one form or another, DevOps is a growing force in the commercial space, so it is important to understand the underlying principles of DevOps to see what role it might have in the the government space.

You can read the rest of the column here and offer your own thoughts here.

Please take a look at my latest column for Government Computing News where I describe why the biggest names in technology are getting involved in the legal battle between Oracle and Google over Android and the profound consequences this case could have for technology in the future. Just to give you an idea, here is the unedited introduction.

In November, the Electronic Frontier Foundation (EFF) filed a brief with the Supreme Court of the United States asking the justices to reverse a decision involving Google and Oracle that would have profound consequences for technology if it stands. To understand why, it is important to look to history for context.

You can read the rest of the column here and offer your own thoughts here.