Article | July 29, 2020
Headquartered in London, England, BP (NYSE: BP) is a multinational oil and gas company. Operating since 1909, the organization offers its customers with fuel for transportation, energy for heat and light, lubricants to keep engines moving, and the petrochemicals products.
Business intelligence has always been a key enabler for improving decision making processes in large enterprises from early days of spreadsheet software to building enterprise data warehouses for housing large sets of enterprise data and to more recent developments of mining those datasets to unearth hidden relationships. One underlying theme throughout this evolution has been the delegation of crucial task of finding out the remarkable relationships between various objects of interest to human beings.
What BI technology has been doing, in other words, is to make it possible (and often easy too) to find the needle in the proverbial haystack if you somehow know in which sectors of the barn it is likely to be. It is a validatory as opposed to a predictory technology.
When the amount of data is huge in terms of variety, amount, and dimensionality (a.k.a. Big Data) and/or the relationship between datasets are beyond first-order linear relationships amicable to human intuition, the above strategy of relying solely on humans to make essential thinking about the datasets and utilizing machines only for crucial but dumb data infrastructure tasks becomes totally inadequate. The remedy to the problem follows directly from our characterization of it: finding ways to utilize the machines beyond menial tasks and offloading some or most of cognitive work from humans to the machines.
Does this mean all the technology and associated practices developed over the decades in BI space are not useful anymore in Big Data age? Not at all. On the contrary, they are more useful than ever: whereas in the past humans were in the driving seat and controlling the demand for the use of the datasets acquired and curated diligently, we have now machines taking up that important role and hence unleashing manifold different ways of using the data and finding out obscure, non-intuitive relationships that allude humans. Moreover, machines can bring unprecedented speed and processing scalability to the game that would be either prohibitively expensive or outright impossible to do with human workforce.
Companies have to realize both the enormous potential of using new automated, predictive analytics technologies such as machine learning and how to successfully incorporate and utilize those advanced technologies into the data analysis and processing fabric of their existing infrastructure. It is this marrying of relatively old, stable technologies of data mining, data warehousing, enterprise data models, etc. with the new automated predictive technologies that has the huge potential to unleash the benefits so often being hyped by the vested interests of new tools and applications as the answer to all data analytical problems.
To see this in the context of predictive analytics, let's consider the machine learning(ML) technology. The easiest way to understand machine learning would be to look at the simplest ML algorithm: linear regression. ML technology will build on basic interpolation idea of the regression and extend it using sophisticated mathematical techniques that would not necessarily be obvious to the causal users. For example, some ML algorithms would extend linear regression approach to model non-linear (i.e. higher order) relationships between dependent and independent variables in the dataset via clever mathematical transformations (a.k.a kernel methods) that will express those non-linear relationship in a linear form and hence suitable to be run through a linear algorithm.
Be it a simple linear algorithm or its more sophisticated kernel methods variation, ML algorithms will not have any context on the data they process. This is both a strength and weakness at the same time. Strength because the same algorithms could process a variety of different kinds of data, allowing us to leverage all the work gone through the development of those algorithms in different business contexts, weakness because since the algorithms lack any contextual understanding of the data, perennial computer science truth of garbage in, garbage out manifests itself unceremoniously here : ML models have to be fed "right" kind of data to draw out correct insights that explain the inner relationships in the data being processed.
ML technology provides an impressive set of sophisticated data analysis and modelling algorithms that could find out very intricate relationships among the datasets they process. It provides not only very sophisticated, advanced data analysis and modeling methods but also the ability to use these methods in an automated, hence massively distributed and scalable ways. Its Achilles' heel however is its heavy dependence on the data it is being fed with. Best analytic methods would be useless, as far as drawing out useful insights from them are concerned, if they are applied on the wrong kind of data. More seriously, the use of advanced analytical technology could give a false sense of confidence to their users over the analysis results those methods produce, making the whole undertaking not just useless but actually dangerous.
We can address the fundamental weakness of ML technology by deploying its advanced, raw algorithmic processing capabilities in conjunction with the existing data analytics technology whereby contextual data relationships and key domain knowledge coming from existing BI estate (data mining efforts, data warehouses, enterprise data models, business rules, etc.) are used to feed ML analytics pipeline. This approach will combine superior algorithmic processing capabilities of the new ML technology with the enterprise knowledge accumulated through BI efforts and will allow companies build on their existing data analytics investments while transitioning to use incoming advanced technologies. This, I believe, is effectively a win-win situation and will be key to the success of any company involved in data analytics efforts.
Article | February 10, 2020
We are a species invested in predicting the future as if our lives depended on it. Indeed, good predictions of where wolves might lurk were once a matter of survival. Even as civilization made us physically safer, prediction has remained a mainstay of culture, from the haruspices of ancient Rome inspecting animal entrails to business analysts dissecting a wealth of transactions to foretell future sales. With these caveats in mind, I predict that in 2020 (and the decade ahead) we will struggle if we unquestioningly adopt artificial intelligence (AI) in predictive analytics, founded on an unjustified overconfidence in the almost mythical power of AI's mathematical foundations. This is another form of the disease of technochauvinism I discussed in a previous article.
Article | April 6, 2020
Today when we look around, we see how technology has revolutionized our world. It has created amazing elements and resources, putting useful intelligence at our fingertips. With all of these revolutions, technology has also made our lives easier, faster, digital and fun. Perhaps at a point when we are talking about technology, Machine learning and artificial intelligence are increasingly popular buzzwords used in modern terms.Machine Learning has proven to be one of the game changer technological advancements of the past decade. In the increasingly competitive corporate world, Machine learning is enabling companies to fast-track digital transformation and move into an age of automation. Some might even argue that AI/ML is required to stay relevant in some verticals, such as digital payments and fraud detection in banking or product recommendations.To understand what machine learning is, it is important to know the concepts of artificial intelligence (AI). It is defined as a program that exhibits cognitive ability similar to that of a human being. Making computers think like humans and solve problems the way we do is one of the main tenets of artificial intelligence.
Article | November 20, 2020
As smart machines, data, and algorithms usher in dramatic technological transformation, its global impact spans from cautious optimism to doomsday scenarios. Widespread transformation, displacement, and disaggregation of world labor markets is speculated in countries like India, with an estimated 600 million workforce by 2022, as well as the global labor market. Even today, we are witnessing the resurgence of 'hybrid' jobs where distinctive human abilities are paired with data and algorithms, and 'super' jobs that involve deep tech. Our historical response to such tectonic shifts and upheavals has been predictable so far - responding with trepidation and uncertainty in the beginning followed by a period of painful transition. Communities and nations that can sense and respond will be able to shape social, economic, and political order decisively. However, with general AI predictably coming of age by 2050-60, governments will need to frame effective policies to respond to their obligations to their citizens. This involves the creation of a new social contract between the individual, enterprise, and state for an inclusive and equitable society.
The present age is marked by automation, augmentation, and amplification of human talent by transformative technologies. A typical career may go through 15-20 transitions. And given the gig economy, the shelf-life of skills is rapidly shrinking. Many agree that for the next 30 years, the nature and the volume of jobs will get significantly redefined. So even as it is nearly impossible to gaze into the crystal ball 100 years later, one can take a shot at what jobs may emerge in the next 20-30 years given the present state. So here is a glimpse into the kind of technological changes the next generation might witness that will change the employment scenario:
RESTORATION OF BIODIVERSITY
Our biodiversity is shrinking frighteningly fast - for both flora and fauna. Extinct species revivalists may be challenged with restoring and reintegrating pertinent elements back into the natural environment. Without biodiversity, humanity will perish.
Medicine is rapidly getting personalized as genome sequencing becomes commonplace. Even today, Elon Musk's Neuralink is working on brain-machine interfaces. So you may soon be able to upload your brain onto a computer where it can be edited, transformed, and re-uploaded back into you. Anti-aging practitioners will be tasked with enhancing human life-spans to ensure we stay productive late into our twilight years. Gene sequencers will help personalize treatments and epigenetic therapists will manipulate gene expression to overcome disease and decay. Brain neurostimulation experts and augmentationists may be commonplace to ensure we are happier, healthier, and disease-free. In fact, happiness itself may get redefined as it shifts from the quality of our relationships to that between man-machine integration.
THE QUANTIFIED SELF
As more of the populace interact and engage with a digitized world, digital rehabilitators will help you detox and regain your sense of self, which may get inseparably intertwined with smart machines and interfaces.
DATA-LED VALUE CREATION
Data is exploding at a torrid pace and becoming a source of value-creation. While today's organizations are scrambling to create data lakes, future data-centers will be entrusted with sourcing high-value data, securing rights to it, and even licensing it to others. Data will increasingly create competitive asymmetries amongst organizations and nations. Data brokers will be the new intermediaries and data detectives, analysts, monitors or watchers, auditors, and frackers will emerge as new-age roles. Since data and privacy issues are entwined together, data regulators, ethicists, and trust professionals will thrive. Many new cyber laws will come into existence.
HEALING THE PLANET
As the world grapples with the specter of climate change, our focus on sustainability and clean energy will intensify. Our landfills are choked with both toxic and non-toxic waste. Plastic alone takes almost 1000 years to degrade, so landfill operators will use earthworm-like robots to help decompose waste and recoup precious recyclable waste. Nuclear fusion will emerge as the new source of clean energy, creating a broad gamut of engineers, designers, integrators, architects, and planners around it. We may even generate power in space. Since our oceans are infested with waste, a lot of initiatives and roles will emerge around cleaning the marine environment to ensure natural habitat and food security.
TAMING THE GENOME
As technologies like CRISPR and Prime-editing mature, we may see a resurgence of biohackers and programmable healthcare. Our health and nutrition may be algorithmically managed. CRISPR-like advancements will need a swathe of engineers, technicians, auditors, and regulators for genetically engineered health that may overcome a wide variety of diseases for longer life-expectancy.
THE RISE OF BOTS
Humanoid and non-humanoid robots will need entire workforce ecosystems around them spanning from suppliers, programmers, operators, and maintenance experts to ethicists and UI-designers. Smart robot psychologists will have to counsel them and ensure they are safe and friendly. Regulators may grant varying levels of autonomy to robots.
DATA LOADS THE GUN, CREATIVITY FIRES THE TRIGGER
Today's deep-learning Generative Adversarial Networks (GANs) can create music like Mozart and paintings like Picasso. Such advancements will give birth to a wide array of AI-enhanced professionals, like musicians, painters, authors, quantum programmers, cybersecurity experts, educators, etc.
FROM AUGMENTATION TO AUTONOMY
Autonomous driving is about to mature in the next few years and will extend to air and space travel. Safety will exceed human capabilities and we may soon reach a state of diminishing returns where we will employ fewer humans to prevent mishaps and unforeseen occurrences. This industry will need supportive command center managers, traffic analyzers, fleet managers, and people to ensure onboarding experience.
BLOCKCHAIN BECOMES PERVASIVE
Blockchain will create a lot of jobs for its mainstream and derivative applications. Even though most of its present applications are in Financial Services, Supply Chain, and Asset Management industries, very soon its adoption and integration will be a lot more expansive. Engineers, designers, UI/UX experts, analysts, auditors, and regulators will be required to manage blockchain-related applications. With Crypto being one of its better-known applications, a lot of transaction specialists, miners, insurers, wealth managers, and regulators will be needed. Crypto exchanges will come under the purview of the regulatory framework.
3D PRINTING TURNS GAME-CHANGER
Additive manufacturing, also popularly called 3D printing, will mature in its precision, capabilities, and market potential. Lab-grown, 3D-printed food will be part of our regular diet. Transplantable organs will be generated using stem cell research and 3D printing. Amputees and the disabled will adopt 3D-printed limbs and prosthetics. Its applications for high-precision reconstructive surgery are already commonplace. Pills are being 3D printed as we speak. So again, we are looking at 3D printers, operators, material scientists, pharmacists, construction experts, etc.
THE COLONIZATION OF OUTER SPACE
Amazon's Blue Origin and Elon Musk's SpaceX signal a new horizon. As space tech gets into a new trajectory, a new breed of commercial space pilots, mission planners, launch managers, cargo experts, ground crew, experience designers, etc. will be required. Since we have ravaged the limited resources of our planet already, mankind will need to venture into asteroid mining for rare and precious metals. This will need scouts and surveyors, meteorologists, remote bot operators, remotely managed factories, and whatnot.
THE HYPER-CONNECTED WORLD
By 2020, we already have anywhere between 50-75 billion connected devices. By 2040, this will likely swell to more than 100 trillion sensors that will spew out a dizzying volume of real-time data ready for analytics and AI. A complete IoT system as we know it is aware, autonomous, and actionable, just like a self-driving car. Imagine the number of data modelers, sensor designers and installers, signal architects and engineers that will be needed. Home automation will be pervasive and smart medicines, implants, and wearables will be the norms of the day.
DRONES USHER IN DISRUPTION
Unmanned aerial and underwater drones are already becoming ubiquitous for applications in aerial surveillance, delivery, and security. Countries are awakening to their potential as well as possibilities of misuse. Command centers, just like that for space travel, will manage them as countries rush to put in a regulatory framework around them. An army of designers, programmers, security experts, traffic flow optimizers will harness their true potential.
SHIELDING YOUR DATA
With data come cyber threats, data breaches, cyber warfare, cyber espionage, and a host of other issues. The more data-dependent and connected the world is, the bigger the problem of cybersecurity will be. The severity of the problem will increase manifold from the current issues like phishing, spyware, malware, viruses and worms, ransomware, DoS/ DDoS attacks, hacktivism, and cybersecurity will indeed be big business. The problem is that threats are increasing 10X faster than investments in this space and the interesting thing is that it is a lot more about audits, governance, policies, and compliance than technology alone.
FOOD-TECH COMES OF AGE
As the world population grows to 9.7 billion people in 2050, cultured food and lab-grown meat will hit our tables to ensure food security. Entire food chains and value delivery networks will see an unprecedented change. Agriculture will be transformed with robotics, IoT, drones, and the food-tech sector will take off in a big way.
QUANTUM COMPUTING SOLVES INTRACTABLE PROBLEMS
Finally, while the list is very long, let’s touch upon the advent of qubits, or Quantum computing. With its ability to break the best encryption on the planet, the traditional asymmetric encryption, public key infrastructure, digital envelopes, and digital certificates in use today will be rendered useless. Bring in the quantum programmers, analysts, privacy and trust managers, health monitors, etc.
As we brace for the world that looms large ahead of us, the biggest enabler that will be transformed itself will be Education 4.0. Education will cease to be a phase in your life. Life-long interventions will be needed to adapt, impart, and shape the skills of individuals that are ready for the future of work. More power to the people!