These days, everyone’s in a hurry. Fast food, fast cars and even faster business processes. It never stops. Do you ever wonder what all the speed is for?
Well, when it comes to data analytics, speed means a lot. It can make the difference between reporting on the consequences of a problem or stopping the problem before it starts.
Turning Up the Speed in a World of Dynamic Data
Picture a traditional data warehouse scenario. You have a data warehouse and SQL or a database, which connects to the API. Processes go through a complex, heavily-scripted ETL to get that data into a warehouse. Then, you connect your BI tool to all of that. It’s not a pretty picture.
That’s because when it comes to high-speed data analytics, BI platforms are seriously challenged. When they load an aggregation of data, they load it into memory at some level. But they’re designed to sit on top of warehouses of relatively static data, not data that changes often or continuously.
With an API, you want to capture data in as close to real time as you can. For example, you might want to access the data flow through the Twitter API, when it’s streaming. But you get access to the data only when it’s created. When it’s gone, it’s gone. For example, if it’s in the Twitter’s database, historically, there are access restrictions through the API. Definitely not helpful.
That’s where tools like TIBCO Spotfire come in. With Spotfire, you can connect your tool to data directly with an API. TIBCO Enterprise Runtime for R (TERR) provides access to an entire suite of online APIs. With them, you can connect to data directly as it populates a website. These web-hosted applications include things like Google or Yahoo Finance APIs, expense trackers or whatever it is that you have.
When you connect directly to data, there are a lot of benefits. You see the data immediately, and there’s no complex uploading process—it’s simply faster.
Unleash the Power of R and Spotfire Automation
Our innovation experts at Syntelli have created a new way to use Spotfire. It refreshes data automatically, not through a website, not through anything external to the application, but only within Spotfire tools.
Using R to Accelerate Financial Data Analytics
Here’s an example of how the tool works. You’ve built a model in R (learn how to make the R language your own), and you want to evaluate it. You’re using commodity prices as transactions appear in your system. The data can include market-level stock or commodity indices as they stream through an API.
You can set Spotfire to refresh data every minute, every second or any interval that you want. Because Spotfire accesses TERR, you can run any script that you want. So, alerts will appear on your dashboard whenever specific aggregates reach specific levels. You can configure email notifications, so Spotfire will send alerts to users whenever a KPI goes above a pre-defined threshold.
This approach opens a world of possibilities and extends the capabilities of tools like Spotfire far beyond those of a simple reporting engine. Tools now become a hub of intelligence for your business versus simple business intelligence.
Moving from BI to Data Analytics
More and more often, we’re seeing BI platforms become analytics platforms. Our clients are adopting analytics capabilities within our business intelligence platforms. And most of these new capabilities are based on R.
I look forward to seeing what our innovation experts at Syntelli come up with as they continue to pursue the extensibility of R in Spotfire and the integration of R into big data projects. Some examples:
- R-based calculations of streaming data as it lands into a big data repository.
- The use of R in other BI platforms.
- Using R in a standalone app such as Shiny in visualizations.
Real-Life Benefits of Real-Time Spotfire
Every task that involves employees keeping an eye on KPIs and reporting when it reaches a certain threshold can be automated (or if nothing else, be made to run faster). When we automate a process, we capture that data right away, gather it from disparate sources, refresh it and fire off an alert if it’s above a pre-defined threshold.
Faster manufacturing QA. In manufacturing job processes move through a plant in stages. You can combine Spotfire’s ability to provide custom geospatial visualizations with its ability to access data directly from the source.
As you refresh those visualizations, you can work with our geographic information specialists. Together, you can create custom shape files of the plant itself and create a heat map that refreshes in near-real-time.
You can do time series calculations of that data to see the total or average time a particular job at a particular location is taking. This gives quality control specialists a single point of truth. They don’t have to inspect every individual unit or crawl through a giant spreadsheet. They have a clean, easy-to-understand visualization. With it, they can keep an eye on their entire plant from one location. It’s rather like a true dashboard that updates constantly as the information arrives.
Faster logistics updates. You can use Spotfire in the same way for logistics. Planes, trains trucks—all of these modes of transport move around the world and ship packages. Every one sends out geospatial coordinates from their sensors. That data is aggregated into a geospatial or time series database or another type of data source that’s accessible to your BI tool.
Ordinarily, it takes a while to load that information. But if you can create an effective real-time scenario, you can plot relevant data in real time on a map. Your platform would grab the most recent information, the starting and ending information in a time series or other significant data and plot it on a graph.
Of course, real-time data analysis is the ideal. In near-real-time situations, there will be delays as data comes from its origin to its final location in its data source. There are bandwidth constraints, including loading it into memory and going through the IO.
We’re getting closer and closer to the real-time data ideal, and these are cases in which near-real-time processing is more than good enough. I wouldn’t suggest using Spotfire for high-frequency trading, which is thoroughly automated. But when you need to know if a truck has gone off course, Spotfire is an excellent way to find out in an acceptable timeframe.
Syntelli is a proud TIBCO Spotfire partner and our data scientists are passionate about solving your organization’s problems! Contact us today or request a demo of Syntelli Services!
Request a Demo!
Daniel Smith
Principal Consultant
About Daniel: Using Business Intelligence platforms to bridge the gap between Advanced Data Analytics and the efficient effective principles of accounting, Daniel applies technology and mathematics to make business faster and smarter.
Daniel has managed solutions for diverse client sectors such as as advertising, military, insurance, and oil & gas. These solutions include Business Intelligence Platform management, online key performance indicator identification and tracking, to full predictive data model construction. Although the analytic solutions are often mathematically complex, Daniel’s presentation and academic background ensures any insights delivered by solutions are relevant and simple to understand.
