Telecom Rating, Analytics and Invoicing Platform (Mining Intelligence from CDRs)

By: Kaustav Majumder

Telephony (Fixed Line and Cellular), since its inception, has been bringing people closer by enabling communication in one form or another. Over the years, the means and modes of communication have evolved, driven by advancements in technology and adoption of new products/platforms created by Telecom OEMs and vendors. In today's world of Mobile Communications, penetration and adoption of Mobile devices has far outnumbered that of fixedline phones. The "Telephony Evolution" which started with voice over fixedline phones connected to Networks manned by switchboard operators, has transformed into High Definition Voice, Messaging (SMS, IP based apps), Rich Communication Services, IP Voice/Video and Data over mobile devices. Riding on the core telephony backbone, many "Value Added Services (VAS)" have also been developed for providing additional features and better User experience. Mobile Communications and Telecommunications as a whole, has evolved expeditiously and it wouldn't be wrong to say that our generation is living in a truly connected world. With all this information (Voice, Video, Messaging and Data) flowing across interconnected networks, what becomes critically important is to have a robust Billing and Invoicing Solution to keep track of and generate appropriate invoices for the type and quantity of services used. For a telecom service provider, its "Billing Solution" is among the most important functions which drives its revenue generation and revenue assurance.

"Micro" Billing Solution

With the rapid changes in mobile communications, Billing and Invoicing Solutions have also been evolving over the years. Proprietary Billing Solutions integrated into the OEM platforms are being replaced by decentralized solutions that are often simple, yet, can handle billing/invoicing loads for various types of traffic. These "Micro" Billing Solutions are lean and are focused towards specific billing/invoicing and Analytical needs of mid and lower tier Telecom providers. The two biggest advantages of having such a solution in place are the incremental cost savings, and the flexibility that such solutions offer when compared to full blown Billing Solutions that have a bunch of features that often do not find a use-case. Once integrated into the Cloud infrastructure of the Telco/provider, operational challenges like rapid deployment, decentralization, security, scalability, etc. are also mitigated. As an analogy to the classical example of "David vs Goliath" these "Micro" Billing Solutions are nimble and can fulfill almost all the primary Invoicing and Analytics needs of an Enterprise at a fraction of the cost when compared to paid commercial Billing software.

CDRs - The "cells" in the Billing ecosystem

"Call Detail Record" also referred to as "CDR" is the smallest information unit in a Billing Solution. Each CDR uniquely identifies an individual transaction which could be either Voice (over traditional or IP Networks), Messages (SMS/MMS), Video or Data. Individual CDRs are usually written into files with each file often containing multiple rows of CDR data. These files are stored and later processed by the Billing Solution. As is evident, it is extremely important that the telecommunications application generating the CDRs must include all relevant fields that are required to generate the billing data/invoices. In most implementations, the CDRs include additional fields that are not directly used for billing purposes. These additional fields contain vital pieces of information that allows the Billing Solution to augment itself and be used for

  • Compliance with law enforcement agencies seeking evidential information
  • Analytics and Reports
  • Monitoring fraud scenarios and track/control excess usage

Design considerations for developing a "Micro" Billing Solution

A Billing Solution should handle the specific Billing/Invoicing needs of the telco/provider where it is deployed. Such a solution should not only handle current requirements but should also be modular enough to accommodate future enhancements. Keeping the scope of the solution in mind, the following considerations should be a part of the design and development process -

Basic requirements:

  • Compliance with law enforcement agencies seeking evidential information
  • Analytics and Reports
  • Monitoring fraud scenarios and track/control excess usage
Software components and considerations:

  • Freeware/Community edition software or licensed software
  • Database for CDR storage and storage of other associated information sets
  • Table partitioning scheme for enhanced database responsiveness
  • Front-end for invoice generation, data manipulation and Analytics/Report generation
User Interface:

1. Should the solution have a dedicated UI of its own?

  • Additional effort towards UI development
  • Recurring effort towards UI enhancements depending on changes implemented
  • An interface, if restrictive in design, might limit the full potential of the solution

2. Should it implement a partially headless design?

  • Make use of command line scripts and job scheduling
  • Make use of available UI features of the individual components of the solution

  • Should be relatively simple to use
  • Should be an automated system requiring minimal manual intervention
  • Should allow execution of dynamic SQL queries for generating Reports and running Analytics
  • Should be fast enough to process huge amount of CDR traffic within reasonable time
Multi-user with Role-based Access Control and Privileges:

  • Finance/Accounting team - (Read/Write) access to invoice generation and retrieval, reports, pricing and margin analysis
  • Sales team - (Read only) access to invoice retrieval, invoice approval, reports, pricing and margin analysis
  • Support & Operations team - (Selective Read/Write) access to CDR traffic, reports and associated tables
  • Billing Admin - (Read/Write) access to complete solution and its configurations

Production setup (An in-house Billing Solution)

Among the most important contributions from the software development community is Open Source software. With the exceptional capabilities and features offered by Open Source software, it is possible to build a complete Billing Solution by integrating Open Source software components along with custom logic that would control the solution and its attributes. The building blocks are available; however, the core logic would still depend on software programming and scripting. All the individual components must be tightly integrated to create an autonomous and seamless solution.

The design considerations and the information presented in the earlier sections of this article, can be used to create an "in-house" Billing Solution from scratch. This solution can be used in a production environment to process live CDR traffic and replace 3rd Party commercial Billing Solutions resulting in significant cost savings for the organization. Such a solution can not only serve as a Billing/Invoicing Platform but can also be used for CDR storage, Invoice storage, generating dynamic reports, partner invoice validations and analytics on Live traffic.

Key attributes of such a Billing Solution -

  • Built over Open Source software components and hence, does not involve any initial or recurring software costs
  • Can processes Millions of CDR records per day for different types of traffic and can be scaled up easily
  • Aggregate CDR traffic from multiple Application instances and serve as a central solution for CDR storage as well as CDR Processing
  • Can be implemented using a "partially headless" design without a specific User interface, making use of native UI provided by its individual software components.
  • Depending on performance requirements, can be implemented over basic compute hardware (desktop) or over server hardware
  • Can be implemented on Windows as well as Linux
  • Can be deployed on premises or can be deployed over a Public cloud
  • Multi-User solution wherein multiple users/teams can work simultaneously to execute tasks such as Invoicing, Reports, Analytics, etc.
  • Can be integrated with Monitoring and Alerting solutions to report suspicious traffic patterns, unusual traffic failures or other such scenarios
Software components that can be used for such a Billing Solution -

  • CDR File Transfer (SFTP) - Cygwin SSHD and CLI scripts
  • ETL and Task Automation - Pentaho Data Integration (Community Edition)
  • Database Server - MySQL (Community Edition)
  • Database Management and Administration - MySQL Workbench (Community Edition)
  • Invoice layout and Invoice file creation - Pentaho Report Designer (Community Edition)
  • Invoicing logic and rules - Intelligent, dynamic queries embedded within Pentaho Report Designer templates
  • Reports and Analytics - Intelligent queries and Stored Procedures written in SQL
  • Task Scheduling - Windows Task Manager / Unix cron jobs (depending on OS platform)

Divide, Design and Conquer (the right way)

If a problem or goal is divided into smaller portions and then each portion is solved, the whole problem gets solved and the goal is achieved. Though this article talks about a Billing Solution, the basic principles for creating a software solution from scratch, or for that matter, solving any problem remains the same - breakdown the problem, design, improve, redesign, integrate, test and deploy. Though often hidden under layers, the core design is crucial as that would support all the current functionality and future enhancements.

The landscape in the already crowded software and technology space, is ever changing and dynamic. Software that is relevant and significant today often gets overshadowed by something innovatively fresh, though, seemingly insignificant. Thus, in current times, innovation, precision and openness to change hold almost an existential significance for software solutions as well as for businesses as a whole. The strength, the core, though, remains with the design. Follow lean, nimble and agile principles in your designs; after all, "David" followed the same principles to defeat "Goliath" thousands of years back . . .

Share This