Interview with Ryan Guthrie by Dan Baker, Black Swan Telecom Journal Up to now, most of the telecom big data action has been happening at the large carriers. TRI’s 2014 study of telecom analytics solutions estimated that merchant market currently at $1,990 million worldwide. As shown in the chart below, over 70% of the big data/analytics solutions are serving the large, giant carriers.
Medium to small operators know “big data” is out there alright, but most have taken only modest steps to use advanced analytics. Technical staffing skills have prevented them from taking advantage of big data advanced analytics. Plus many are not quite sure what practical benefits big data brings them.
Well, good news: cloud services are emerging to enable smaller operators to get started and avoid the cost of hiring a team of data scientists.
A pioneer in this big data cloud service for telecoms is New Jersey-based Advanced Technologies & Services (ATS), a revenue assurance and analytics specialty firm celebrating two decades of business this year. Ryan Guthrie, the company’s VP of Solutions Engineering now joins us to explain what ATS’ service is all about.
Dan Baker: Ryan, it’s great to hear that somebody is looking out for the smaller operators who are often left in the dust by new technology and innovation. What’s the biggest need for these operators as they start using analytics in the cloud?
Ryan Guthrie: Dan, believe it or not, one of the key things people struggle with is storage. There’s a requirement to keep five years worth of call detail records (CDRs), but that ends up being a bit of a challenge. Folks end up storing it on external hard drives under their desks and they waste a lot of time managing that and it’s hard to find data when you’re looking for it.
You’re better off storing the data in the cloud where it’s searchable, sortable, and they can very quickly access it.
That’s the service we provide. We use Amazon Web Services and some Google services like BigQuery and BigTable. These work similar to MySQL, but they are much faster. Now a power user can run SQL queries on the data, or we can create canned reports for people who are looking for the same information on a daily/weekly basis.
So our market for this cloud service stretches from the Tier 1 carriers to the hundreds of small to medium sized operators in North America. The flexibility of the cloud allows our pricing model to scale for carriers of all sizes. That means cost is no longer a barrier for smaller operators to get started with big data.
Baker: What got you interested in offering this kind of service? What’s your company’s background in big data?
Guthrie: A few years ago, we experimented with Hadoop technology, a way of processing data using scalable, commodity hardware. We created our own Hadoop cluster in our shop. Trouble is, it takes a lot of technical knowledge to manage Hadoop and we are telecom data experts, not Hadoop experts.
So, in search of an easier to maintain Hadoop environment, we found Amazon Web Services (AWS) and they had exactly the kind of service we needed to allow us to scale up. So we run Hadoop using their backend and it’s seamless as to how many servers they are running. You simply load the data up there and run queries on it.
If you want the results back the next couple of minutes, you can request more servers, more processors assigned to it. If it’s a job you kick off at midnight and get a report the next morning, you can assign less processors to it and end up paying a little bit less. So, it is pretty cool because you more or less pay as you go for processing power.
The other beauty, of course, is you don’t have to worry about buying more servers or figuring out what to do when another hard drive goes down.
ATS Architecture for Big Data Services in the Cloud
Baker: On top of the cloud storage capability, what’s the most popular Big Data application you deliver?
Guthrie: On top of the cloud storage, we offer a variety of modules that can be easily bolted on. [See chart above for typical services] The most popular module is our KPI Dashboard with alarming capabilities. We provide a complete turnkey solution that monitors call volumes and is built on a statistical model to alarm when volumes spike/dip outside of what’s expected.
At any time you can look and see; at 2 o’clock on Thursday, what volume of traffic do we expect to see? And if the traffic breaks the upper or lower thresholds, an alarm is triggered. So if there was a big spike, maybe it’s fraud. If there’s a big drop, then it may be that the switch went down or some records didn’t come off the switch.
To populate the dashboard, we typically get a copy of the CDRs right off the switch and it’s loaded into the system automatically as the data is sent. The switch cuts the data and sends one copy of those records to the company’s mediation system and a copy gets sent to us.
Baker: What other applications are available?
Guthrie: Well, another popular one is retrieval of the phone records for law enforcement. So we supply what we call our subpoena management module that maintains and finds the data needed to support Federal, State, and local law enforcement efforts when you need to retrieve phone calls and the like.
Some of the medium carriers get daily subpoena requests. The smaller operators might get a request once a week, and having a cloud solution really simplifies this process.
We also have a margin analysis module that’s useful in wholesale, primarily monitoring international voice traffic. Companies negotiate different deals over time and for one reason or another those routes start losing money. If you’re not on top of it, it can become a big revenue leakage point.
Baker: What about security in the cloud? I’m sure that’s a question you get asked quite often.
Guthrie: You are absolutely right. One of the most common questions we get about storing data in the cloud is: Will my data be safe? The answer is yes, the data is just as safe, if not safer, than how they currently store data today. Cloud services such as AWS understand how important security is to its clients and they do a great job at providing it. If you look at who is using AWS to store data today it includes big name companies like Netflix, Comcast, Nokia, and even federal agencies like the CIA, FBI, and NASA. You can be sure that if the CIA trusts the security of the cloud, you can too.
Baker: Ryan, it sounds like a very useful and much needed service to get the smaller operators rolling with a big data solution that’s practical and affordable.