Harnessing the Value of Big Data Analytics

| June 13, 2012

article image
In business publications and IT trade journals, the buzz about “big data” challenges is nearly deafening. Rapidly growing volumes of data – from transactional systems like enterprise resource planning (ERP) software and non-transactional sources such as web logs, customer call center records, and video images – are everywhere. A tsunami of data, some experts call it. Most companies know how to collect, store, and analyze their operational data. But these new multi-structured data types are often too variable and dynamic to be costeffectively captured in a traditional data schema using only a standard query language (SQL) for analytics.

Spotlight

Starburst

Starburst gives analysts the freedom to work with diverse data sets wherever their location, without compromising on performance. Using Presto, the Starburst Distribution of Presto provides fast, interactive query performance across a wide variety of data sources including HDFS, Amazon S3, MySQL, SQL Server, PostgreSQL, Cassandra, MongoDB, Kafka, and Teradata, among others. Founded by the largest team of Presto committers outside of Facebook, Starburst is the only company providing enterprise support for the Presto project. Originally invented and open sourced by Facebook under the Apache License, Presto is the fastest growing SQL engine driven by a community of users --- from small companies to the Fortune 500.

OTHER ARTICLES

How big data can help the homeless

Article | March 12, 2020

Homeless policy needs to join the big data revolution. A data tsunami is transforming our world. Ninety percent of existing data was created in the last two years, and Silicon Valley is leveraging it with powerful analytics to create self-driving cars and to revolutionize business decision-making in ways that drive innovation and efficiency.Unfortunately, this revolution has yet to help the homeless. It is not due to a lack of data. Sacramento alone maintains data on half a million service interactions with more than 65,000 homeless individuals. California is considering integrating the data from its 44 continuums of care to create a richer pool of data. Additionally, researchers are uncovering troves of relevant information in educational and social service databases.These data, however, are only useful if they are aggressively mined for insights, looking for problems to solve and successful practices to replicate. At that juncture California falls short.

Read More

Machine Learning and AI is Supercharging the Modern Technology

Article | April 6, 2020

Today when we look around, we see how technology has revolutionized our world. It has created amazing elements and resources, putting useful intelligence at our fingertips. With all of these revolutions, technology has also made our lives easier, faster, digital and fun. Perhaps at a point when we are talking about technology, Machine learning and artificial intelligence are increasingly popular buzzwords used in modern terms.Machine Learning has proven to be one of the game changer technological advancements of the past decade. In the increasingly competitive corporate world, Machine learning is enabling companies to fast-track digital transformation and move into an age of automation. Some might even argue that AI/ML is required to stay relevant in some verticals, such as digital payments and fraud detection in banking or product recommendations.To understand what machine learning is, it is important to know the concepts of artificial intelligence (AI). It is defined as a program that exhibits cognitive ability similar to that of a human being. Making computers think like humans and solve problems the way we do is one of the main tenets of artificial intelligence.

Read More

Natural Language Desiderata: Understanding, explaining and interpreting a model.

Article | May 3, 2021

Clear conceptualization, taxonomies, categories, criteria, properties when solving complex real-life contextualized problems is non-negotiable, a “must” to unveil the hidden potential of NPL impacting on the transparency of a model. It is common knowledge that many authors and researchers in the field of natural language processing (NLP) and machine learning (ML) are prone to use explainability and interpretability interchangeably, which from the start constitutes a fallacy. They do not mean the same, even when looking for a definition from different perspectives. A formal definition of what explanation, explainable, explainability mean can be traced to social science, psychology, hermeneutics, philosophy, physics and biology. In The Nature of Explanation, Craik (1967:7) states that “explanations are not purely subjective things; they win general approval or have to be withdrawn in the face of evidence or criticism.” Moreover, the power of explanation means the power of insight and anticipation and why one explanation is satisfactory involves a prior question why any explanation at all should be satisfactory or in machine learning terminology how a model is performant in different contextual situations. Besides its utilitarian value, that impulse to resolve a problem whether or not (in the end) there is a practical application and which will be verified or disapproved in the course of time, explanations should be “meaningful”. We come across explanations every day. Perhaps the most common are reason-giving ones. Before advancing in the realm of ExNLP, it is crucial to conceptualize what constitutes an explanation. Miller (2017) considered explanations as “social interactions between the explainer and explainee”, therefore the social context has a significant impact in the actual content of an explanation. Explanations in general terms, seek to answer the why type of question. There is a need for justification. According to Bengtsson (2003) “we will accept an explanation when we feel satisfied that the explanans reaches what we already hold to be true of the explanandum”, (being the explanandum a statement that describes the phenomenon to be explained (it is a description, not the phenomenon itself) and the explanan at least two sets of statements, used for the purpose of elucidating the phenomenon). In discourse theory (my approach), it is important to highlight that there is a correlation between understanding and explanation, first and foremost. Both are articulated although they belong to different paradigmatic fields. This dichotomous pair is perceived as a duality, which represents an irreducible form of intelligibility. When there are observable external facts subject to empirical validation, systematicity, subordination to hypothetic procedures then we can say that we explain. An explanation is inscribed in the analytical domain, the realm of rules, laws and structures. When we explain we display propositions and meaning. But we do not explain in a vacuum. The contextual situation permeates the content of an explanation, in other words, explanation is an epistemic activity: it can only relate things described or conceptualized in a certain way. Explanations are answers to questions in the form: why fact, which most authors agree upon. Understanding can mean a number of things in different contexts. According to Ricoeur “understanding precedes, accompanies and swathes an explanation, and an explanation analytically develops understanding.” Following this line of thought, when we understand we grasp or perceive the chain of partial senses as a whole in a single act of synthesis. Originally, belonging to the field of the so-called human science, then, understanding refers to a circular process and it is directed to the intentional unit of discourse whereas an explanation is oriented to the analytical structure of a discourse. Now, to ground any discussion on what interpretation is, it is crucial to highlight that the concept of interpretation opposes the concept of explanation. They cannot be used interchangeably. If considered as a unit, they composed what is called une combinaison éprouvé (a contrasted dichotomy). Besides, in dissecting both definitions we will see that the agent that performs the explanation differs from the one that produce the interpretation. At present there is a challenge of defining—and evaluating—what constitutes a quality interpretation. Linguistically speaking, “interpretation” is the complete process that encompasses understanding and explanation. It is true that there is more than one way to interprete an explanation (and then, an explanation of a prediction) but it is also true that there is a limited number of possible explanations if not a unique one since they are contextualized. And it is also true that an interpretation must not only be plausible, but more plausible than another interpretation. Of course there are certain criteria to solve this conflict. And to prove that an interpretation is more plausible based on an explanation or the knowledge could be related to the logic of validation rather than to the logic of subjective probability. Narrowing it down How are these concepts transferred from theory to praxis? What is the importance of the "interpretability" of an explainable model? What do we call a "good" explainable model? What constitutes a "good explanation"? These are some of the many questions that researchers from both academia and industry are still trying to answer. In the realm on machine learning current approaches conceptualize interpretation in a rather ad-hoc manner, motivated by practical use cases and applications. Some suggest model interpretability as a remedy, but only a few are able to articulate precisely what interpretability means or why it is important. Hence more, most in the research community and industry use this term as synonym of explainability, which is certainly not. They are not overlapping terms. Needless to say, in most cases technical descriptions of interpretable models are diverse and occasionally discordant. A model is better interpretable than another model if its decisions are easier for a human to comprehend than decisions from the other model (Molnar, 2021). For a model to be interpretable (being interpretable the quality of the model), the information conferred by an interpretation may be useful. Thus, one purpose of interpretations may be to convey useful information of any kind. In Molnar’s words the higher the interpretability of a machine learning model, the easier it is for someone to comprehend why certain decisions or predictions have been made.” I will make an observation here and add “the higher the interpretability of an explainable machine learning model”. Luo et. al. (2021) defines “interpretability as ‘the ability [of a model] to explain or to present [its predictions] in understandable terms to a human.” Notice that in this definition the author includes “understanding” as part of the definition, giving the idea of completeness. Thus, the triadic closure explanation-understanding-interpretation is fulfilled, in which the explainer and interpretant (the agents) belong to different instances and where interpretation allows the extraction and formation of additional knowledge captured by the explainable model. Now are the models inherently interpretable? Well, it is more a matter of selecting the methods of achieving interpretability: by (a) interpreting existing models via post-hoc techniques, or (b) designing inherently interpretable models, which claim to provide more faithful interpretations than post-hoc interpretation of blackbox models. The difference also lies in the agency –like I said before– , and how in one case interpretation may affect the explanation process, that is model’s inner working or just include natural language explanations of learned representations or models.

Read More

How Business Analytics Accelerates Your Business Growth

Article | April 30, 2020

In the present complex and volatile market with data as a nucleus, analytics becomes a core function for any enterprise that relies on data-driven insights to understand their customers, trends, and business environments. In the age of digitization and automation, it is only sensible to make a move to analytics for a data-driven approach for your business. While a host of sources including Digital clicks, social media, POS terminal, and sensors enrich the data quality, data can be collected along various stages of interactions, and initiatives were taken. Customers leave their unique data fingerprint when interacting with the enterprise, which when put through analytics provides actionable insights to make important business decisions. Table of Contents: Business Analytics or Business Intelligence: The Difference Growth Acceleration with Business Analytics Business Analytics or Business Intelligence (BI): The Difference Business Intelligence comes within the descriptive phase of analytics. BI is where most enterprises start using an analytics program. BI uses software and services to turn data into actionable intelligence that helps an enterprise to make informed and strategic decisions. It’s information about the data itself. It’s not trying to do anything beyond telling a story about what the data is saying. - Beverly Wright, Executive Director, Business Analytics Center, Georgia Tech’s Scheller College of Business Some businesses might use BI and BA interchangeably, though some believe BI to be the know-how of what has happened, while the analytics or advanced analytics work to anticipate the various future scenarios. BI uses more structured data from traditional enterprise platforms, such as enterprise resource planning (ERP) or financial software systems, and it delivers views into past financial transactions or other past actions in areas such as operations and the supply chain. Today, experts say BI’s value to organizations is derived from its ability to provide visibility into such areas and business tasks, including contractual reconciliation. Someone will look at reports from, for example, last year’s sales — that’s BI — but they’ll also get predictions about next year’s sales — that’s business analytics — and then add to that a what-if capability: What would happen if we did X instead of Y. - CindiHowson, research vice president at Gartner A subset of business intelligence (BI), business analytics is implemented to determine which datasets are useful and how they can be leveraged to solve problems and increase efficiency, productivity, and revenue. It is the process of collating, sorting, processing, and studying business data, and using statistical models and iterative methodologies to transform data into business insights. BA is more prescriptive and uses methods that can analyze data, recognize patterns, and develops models that clarify past events, make future predictions, and recommend future discourse. Analysts use sophisticated data, quantitative analysis, and mathematical models to provide a solution for data-driven issues. To expand their understanding of complex data sets, and artificial intelligence, deep learning, and neural networks to micro-segment available data and identify patterns they can utilize statistics, information systems, computer science, and operations research. Let’s discuss the 5 ways business analytics can help you accelerate your business growth. READ MORE: HOW TO OVERCOME CHALLENGES IN ADOPTING DATA ANALYTICS Growth Acceleration with Business Analytics 1. Expansion planning Let’s say you’re planning an expansion opening a branch, store, restaurant, or office in a new location and have accumulated a lot of information about your growing customer base, equipment or other asset maintenance, employee payment, and delivery or distribution schedule. What if we told it is possible to get into a much detailed planning process with all that information available? It becomes possible with business analytics. With BA you can find insights in visualizations and dashboards and then research them further with business intelligence and reports. Moreover, you can interact with the results and use the information to create your expansion plan. 2. Finding your audience You’re right to examine your current customer data but you should also be looking into the customer sentiments towards your brand and who is saying what, and in what parts of the region. Business Analytics offers social media analysis so you can bring together internal and external customer data to create a profile of your customers, both existing and potential. Thus, you have prepared an ideal demographic, which can be used to identify people that are most likely to turn to your products or services. As a result, you have successfully deduced the area that offers the most in terms of expansion and customer potential. 3. Creating your business plan The real-time interaction with your data provides a detailed map of the current progress as well as your performance. Business Analytics solutions offer performance indicators to find and forecast trends in sales, turnover, and growth. This can be used in the in-depth development of a business plan for the next phase of your thriving franchise. 4. Developing your marketing campaign With Business Analytics, you’re capable of sending the right message to the audience most eager to try your product/service as part of a marketing campaign. You’re empowered to narrow down branding details, messaging tone and customer preferences, like the right offers that will differentiate you from the other businesses in the area. Using BA, you have gained a competitive edge by making sure you offer something new to your customers and prospects. It enables you to use your data to derive customer insights, make insight-driven decisions, do targeted marketing, and make business development decisions with confidence. 5. Use predictive insights to take action With analytics tools like predictive analytics, your expansion plans are optimized. It enables you to pinpoint and research about the factors that are influencing your outcomes so that you can be assured of being on the right track. When you can identify and understand your challenges quickly and resolve them faster, you improve the overall business performance resulting in successful expansion and accelerated growth. READ MORE: WHAT IS THE DIFFERENCE BETWEEN BUSINESS INTELLIGENCE, DATA WAREHOUSING AND DATA ANALYTICS

Read More

Spotlight

Starburst

Starburst gives analysts the freedom to work with diverse data sets wherever their location, without compromising on performance. Using Presto, the Starburst Distribution of Presto provides fast, interactive query performance across a wide variety of data sources including HDFS, Amazon S3, MySQL, SQL Server, PostgreSQL, Cassandra, MongoDB, Kafka, and Teradata, among others. Founded by the largest team of Presto committers outside of Facebook, Starburst is the only company providing enterprise support for the Presto project. Originally invented and open sourced by Facebook under the Apache License, Presto is the fastest growing SQL engine driven by a community of users --- from small companies to the Fortune 500.

Events