Pivotal Query Optimizer for Greenplum Database: Overview and Demo

| May 18, 2016

article image
The query optimizer available in Greenplum Database (open source project name: GPORCA) is the industry’s first cost-based query optimizer for big data workloads. It can scale interactive and batch mode analytics to large datasets in the petabytes without degrading query performance and throughput.

Spotlight

High Tech Engineering Center

HTEC (High Tech Engineering Center) is a center of innovation and software development. At HTEC, award-winning creativity sits alongside deep engineering skill. Founded in September 2008 and headquartered in San Francisco, HTEC is a global company with development centers comprising over 300 engineers across Serbia as well as local offices in the United Kingdom, Sweden and China.

OTHER ARTICLES

CYBERSECURITY STRATEGIES TO MAKE IT NETWORKS MORE RESILIENT TO CYBERATTACKS

Article | February 28, 2020

The increasing use of advanced technologies and the internet have created an attack surface for malicious attackers. With these progressions, businesses’ IT systems are now more vulnerable which has led them to leverage innovative cybersecurity strategies that can thwart and make their networks more resilient to cyberattacks. Cybercriminals can use a variety of attacks against individuals or businesses like accessing, changing or deleting sensitive data; extracting payment; interfering with business processes and more.These kinds of attacks present an evolving danger to organizations, employees and consumers, and can cost them reputation, finances and personal lives to some extent. So, in order to protect IT networks from cyberattacks, it is significant to be aware of the various aspects of cybersecurity.

Read More

7 Data Storage Trends You Cannot Miss in a Data Center

Article | July 23, 2020

Contents: 1 Introduction 2 Top Data Storage Trends That Simplify Data Management 2.1 AI Storage Continues to be The Chief 2.2 Price Markdown in Flash Storage 2.3 Hybrid Multi Cloud for The Win 2.4 Increased Significance of Software-Defined Storage 2.5 Non-Volatile Memory Express (NVMe) Beats Data Center Fabrics 2.6 Acceleration of Storage Class Memory 2.7 Hyperconverged Storage – A Push to Edge Computing 3 The Future of Data Storage 1. Introduction There’s more to data than just to store it. Organizations not only have the responsibility of dealing with a plethora of data, but are also anticipated of safeguarding it. One of the primary alternatives that enterprises are indulging in to keep up with the continuous data expansion is data storage entities and applications. A recent study conducted by Statista revealed that worldwide spending on data storage units is expected to exceed 78 billion U.S. dollars by 2021. Going by these storage stats, it can be certainly said that data is going to be amplified at a much faster rate, and companies do not have a choice but to be geared up for a data boom and still be relevant. When it comes to data management/storage, information technology has risen to all its glory with concepts like machine learning. While the idea of such profound approaches is thrilling, the real question boils down to whether organizations are ready as well as equipped enough to handle them. The answer to this might be NO. But, can companies make changes and still thrive? Most definitely, YES! To make this concept more understandable, here is a list of changes/trends that companies should adopt to make data storage a lot more easy and secure. 2. Top data storage trends that simplify data management Data corruption is one big issue that most companies face. The complications that unfold further because of the corruption of data are even more complicated to resolve. To fix this and other such data storage problems, companies have come up with trends that are resilient and flexible. These trends have the capability of making history in the world of technology, so, you better gear up to learn and later adapt to them. 2.1 AI storage continues to be the chief The speed with which AI hit the IT world just doesn’t seem to slow down even after all these years. We say this because, amongst all other concepts that were and are constantly being introduced, artificial intelligence is one applied science that has made the most amount of innovations. To further add to this, AI is now making enterprise data storage process easier with its various subsets like machine learning and deep learning. This technology is helping companies in accumulating multiple layers of data in a more assorted format. It is automating IT storages including data migrating, archiving, protecting, etc. With AI, companies will be able to control data storage across multiple locations and storage platforms. 2.2 Price markdown in Flash storage As per a report by Markets and Markets, the overall All-Flash Array Market was valued at USD 5.9 billion in 2018 and is expected to reach USD 17.8 billion by 2023, at a CAGR of 24.53% during this period. This growth only states that the need for all-flash storage is only going to broaden. Flash storage has always been a choice that most companies stayed away from mainly because of the price. But with this new trend of adopting flexible data storage ways coming in, flash storage has been offered at a much-depreciated price. The drop in the cost of this storage technology will finally enable businesses of all sizes to invest in this high-performance solution. READ MORE: HOW BUSINESS ANALYTICS ACCELERATES YOUR BUSINESS GROWTH 2.3 Hybrid multi cloud for the win With data growing every minute, just a “cloud” strategy will not be enough. In this wave of data storage services, hybrid multi-cloud is one concept that is helping manage off-premises data. With this growing concept, IT authorities will be able to collect, segregate and store, on-premises, and off-premises data in a much-sophisticated manner. This will enable in centrally managing while reducing the effort of data storage by automating policy-based data placement across a hybrid of multi-cloud and storage types. 2.4 Increased significance of software-defined storage More the data, less reliability on hardware devices – this is the growing attitude of most companies. This fear certainly has the possibility of becoming a reality. Hence, an addition to the cybersecurity strategy that companies can make is adopting software-defined storage. This approach of data storage disconnects the underlying physical storage hardware. It is programmed in a way that can function on policy-based management of resources, automated provision, and computerized storage capacity reassignment. Due to the automated function, scaling up and down of data is also faster. Some of the biggest advantages of this trend will be the governance, data protection, and security it will provide to the entire loop. 2.5 Non-Volatile Memory Express (NVMe) beats data center fabrics NVMe – as ornate as the name sounds, is a concept that is freshly introduced with the aim of making data storage simpler. Non-Volatile Memory Express is a concept that enables accessibility of high-speed storage media. It is a protocol that is showing great results in a short amount of time of its inception. NVMe not only increases the performance value of existing applications, but also enables new applications to real-time workload processing. This feature of high performance and low latency is surely a highlight of the concept. All in all, this entire trend seems to have a lot of potential that are yet to be explored. READ MORE: HOW TO MAXIMIZE VALUE FROM DATA COLLECTED FOR BUSINESSES SUCCESS 2.6 Acceleration of storage class memory Storage class memory is a perfect combination of flash storage and NVMe. This is because it perfectly fills in the gap between server storage and external storage. As data protection is one of the major concerns of enterprises, this upcoming trend, does not only protect data but also continually stores and improves it for easier segregation. A clear advantage that storage class memory has over flash and NVMe storages is that it provides memory-like byte-addressable access to data thus reducing piling up of irrelevant data. Another benefit of this trend is that it indulges in deeper integration of data for ensuring high performance and top-level data security. 2.7 Hyperconverged storage – a push to edge computing The increased demand for hyper converged storage is a result of the growth of hybrid cloud and software-defined infrastructure. Besides these technologies, its suitability for retail settings and remote offices is add on to its already existing set of features. It’s the capability of capturing data from a distance also enables cost-effectiveness and scales down the need to store everything on a public cloud. Hyper converged storage if used in its true essence can simplify IT operations and data storage for enterprises of all sizes. 3. The future of data storage According to the Internet World Stats, more than 4.5 billion internet users around the world relentlessly create an astronomical amount of data. This translates to propel companies into discovering methods or applications that help them store this data safe from harmful ransomware attacks and still use it productively for their advantage. One of the prime changes that can be estimated about the future of data storage is that companies will have to adapt to the rapid changes, and mould their process to enable quick and seamless storage of data. Another enhancement would be that IT managers and responsible authorities would have to be updated and proactive at all times to know what data storage has been newly introduced, and how it can be used for the company’s advantage. Here’s a thing, amongst all the research that enterprises are conducting, not all data storage technologies will end up becoming a hit, and will fulfil the specification of high-speed storage. But, looking at all the efforts that researchers are taking, we don’t think they are going to stop any sooner and neither is the augmentation of data!

Read More

THE NOT-SO-DISTANT FUTURE OF WORK

Article | November 20, 2020

As smart machines, data, and algorithms usher in dramatic technological transformation, its global impact spans from cautious optimism to doomsday scenarios. Widespread transformation, displacement, and disaggregation of world labor markets is speculated in countries like India, with an estimated 600 million workforce by 2022, as well as the global labor market. Even today, we are witnessing the resurgence of 'hybrid' jobs where distinctive human abilities are paired with data and algorithms, and 'super' jobs that involve deep tech. Our historical response to such tectonic shifts and upheavals has been predictable so far - responding with trepidation and uncertainty in the beginning followed by a period of painful transition. Communities and nations that can sense and respond will be able to shape social, economic, and political order decisively. However, with general AI predictably coming of age by 2050-60, governments will need to frame effective policies to respond to their obligations to their citizens. This involves the creation of a new social contract between the individual, enterprise, and state for an inclusive and equitable society. The present age is marked by automation, augmentation, and amplification of human talent by transformative technologies. A typical career may go through 15-20 transitions. And given the gig economy, the shelf-life of skills is rapidly shrinking. Many agree that for the next 30 years, the nature and the volume of jobs will get significantly redefined. So even as it is nearly impossible to gaze into the crystal ball 100 years later, one can take a shot at what jobs may emerge in the next 20-30 years given the present state. So here is a glimpse into the kind of technological changes the next generation might witness that will change the employment scenario: RESTORATION OF BIODIVERSITY Our biodiversity is shrinking frighteningly fast - for both flora and fauna. Extinct species revivalists may be challenged with restoring and reintegrating pertinent elements back into the natural environment. Without biodiversity, humanity will perish. PERSONALIZED HEALTHCARE Medicine is rapidly getting personalized as genome sequencing becomes commonplace. Even today, Elon Musk's Neuralink is working on brain-machine interfaces. So you may soon be able to upload your brain onto a computer where it can be edited, transformed, and re-uploaded back into you. Anti-aging practitioners will be tasked with enhancing human life-spans to ensure we stay productive late into our twilight years. Gene sequencers will help personalize treatments and epigenetic therapists will manipulate gene expression to overcome disease and decay. Brain neurostimulation experts and augmentationists may be commonplace to ensure we are happier, healthier, and disease-free. In fact, happiness itself may get redefined as it shifts from the quality of our relationships to that between man-machine integration. THE QUANTIFIED SELF As more of the populace interact and engage with a digitized world, digital rehabilitators will help you detox and regain your sense of self, which may get inseparably intertwined with smart machines and interfaces. DATA-LED VALUE CREATION Data is exploding at a torrid pace and becoming a source of value-creation. While today's organizations are scrambling to create data lakes, future data-centers will be entrusted with sourcing high-value data, securing rights to it, and even licensing it to others. Data will increasingly create competitive asymmetries amongst organizations and nations. Data brokers will be the new intermediaries and data detectives, analysts, monitors or watchers, auditors, and frackers will emerge as new-age roles. Since data and privacy issues are entwined together, data regulators, ethicists, and trust professionals will thrive. Many new cyber laws will come into existence. HEALING THE PLANET As the world grapples with the specter of climate change, our focus on sustainability and clean energy will intensify. Our landfills are choked with both toxic and non-toxic waste. Plastic alone takes almost 1000 years to degrade, so landfill operators will use earthworm-like robots to help decompose waste and recoup precious recyclable waste. Nuclear fusion will emerge as the new source of clean energy, creating a broad gamut of engineers, designers, integrators, architects, and planners around it. We may even generate power in space. Since our oceans are infested with waste, a lot of initiatives and roles will emerge around cleaning the marine environment to ensure natural habitat and food security. TAMING THE GENOME As technologies like CRISPR and Prime-editing mature, we may see a resurgence of biohackers and programmable healthcare. Our health and nutrition may be algorithmically managed. CRISPR-like advancements will need a swathe of engineers, technicians, auditors, and regulators for genetically engineered health that may overcome a wide variety of diseases for longer life-expectancy. THE RISE OF BOTS Humanoid and non-humanoid robots will need entire workforce ecosystems around them spanning from suppliers, programmers, operators, and maintenance experts to ethicists and UI-designers. Smart robot psychologists will have to counsel them and ensure they are safe and friendly. Regulators may grant varying levels of autonomy to robots. DATA LOADS THE GUN, CREATIVITY FIRES THE TRIGGER Today's deep-learning Generative Adversarial Networks (GANs) can create music like Mozart and paintings like Picasso. Such advancements will give birth to a wide array of AI-enhanced professionals, like musicians, painters, authors, quantum programmers, cybersecurity experts, educators, etc. FROM AUGMENTATION TO AUTONOMY Autonomous driving is about to mature in the next few years and will extend to air and space travel. Safety will exceed human capabilities and we may soon reach a state of diminishing returns where we will employ fewer humans to prevent mishaps and unforeseen occurrences. This industry will need supportive command center managers, traffic analyzers, fleet managers, and people to ensure onboarding experience. BLOCKCHAIN BECOMES PERVASIVE Blockchain will create a lot of jobs for its mainstream and derivative applications. Even though most of its present applications are in Financial Services, Supply Chain, and Asset Management industries, very soon its adoption and integration will be a lot more expansive. Engineers, designers, UI/UX experts, analysts, auditors, and regulators will be required to manage blockchain-related applications. With Crypto being one of its better-known applications, a lot of transaction specialists, miners, insurers, wealth managers, and regulators will be needed. Crypto exchanges will come under the purview of the regulatory framework. 3D PRINTING TURNS GAME-CHANGER Additive manufacturing, also popularly called 3D printing, will mature in its precision, capabilities, and market potential. Lab-grown, 3D-printed food will be part of our regular diet. Transplantable organs will be generated using stem cell research and 3D printing. Amputees and the disabled will adopt 3D-printed limbs and prosthetics. Its applications for high-precision reconstructive surgery are already commonplace. Pills are being 3D printed as we speak. So again, we are looking at 3D printers, operators, material scientists, pharmacists, construction experts, etc. THE COLONIZATION OF OUTER SPACE Amazon's Blue Origin and Elon Musk's SpaceX signal a new horizon. As space tech gets into a new trajectory, a new breed of commercial space pilots, mission planners, launch managers, cargo experts, ground crew, experience designers, etc. will be required. Since we have ravaged the limited resources of our planet already, mankind will need to venture into asteroid mining for rare and precious metals. This will need scouts and surveyors, meteorologists, remote bot operators, remotely managed factories, and whatnot. THE HYPER-CONNECTED WORLD By 2020, we already have anywhere between 50-75 billion connected devices. By 2040, this will likely swell to more than 100 trillion sensors that will spew out a dizzying volume of real-time data ready for analytics and AI. A complete IoT system as we know it is aware, autonomous, and actionable, just like a self-driving car. Imagine the number of data modelers, sensor designers and installers, signal architects and engineers that will be needed. Home automation will be pervasive and smart medicines, implants, and wearables will be the norms of the day. DRONES USHER IN DISRUPTION Unmanned aerial and underwater drones are already becoming ubiquitous for applications in aerial surveillance, delivery, and security. Countries are awakening to their potential as well as possibilities of misuse. Command centers, just like that for space travel, will manage them as countries rush to put in a regulatory framework around them. An army of designers, programmers, security experts, traffic flow optimizers will harness their true potential. SHIELDING YOUR DATA With data come cyber threats, data breaches, cyber warfare, cyber espionage, and a host of other issues. The more data-dependent and connected the world is, the bigger the problem of cybersecurity will be. The severity of the problem will increase manifold from the current issues like phishing, spyware, malware, viruses and worms, ransomware, DoS/ DDoS attacks, hacktivism, and cybersecurity will indeed be big business. The problem is that threats are increasing 10X faster than investments in this space and the interesting thing is that it is a lot more about audits, governance, policies, and compliance than technology alone. FOOD-TECH COMES OF AGE As the world population grows to 9.7 billion people in 2050, cultured food and lab-grown meat will hit our tables to ensure food security. Entire food chains and value delivery networks will see an unprecedented change. Agriculture will be transformed with robotics, IoT, drones, and the food-tech sector will take off in a big way. QUANTUM COMPUTING SOLVES INTRACTABLE PROBLEMS Finally, while the list is very long, let’s touch upon the advent of qubits, or Quantum computing. With its ability to break the best encryption on the planet, the traditional asymmetric encryption, public key infrastructure, digital envelopes, and digital certificates in use today will be rendered useless. Bring in the quantum programmers, analysts, privacy and trust managers, health monitors, etc. As we brace for the world that looms large ahead of us, the biggest enabler that will be transformed itself will be Education 4.0. Education will cease to be a phase in your life. Life-long interventions will be needed to adapt, impart, and shape the skills of individuals that are ready for the future of work. More power to the people!

Read More

Data Analytics: Five use cases in telecom industry

Article | May 27, 2021

The telecom industry has witnessed spectacular growth since its establishment in the 1830s. Enabling distant communications, collaborations, and transactions globally, telecommunication plays a significant role in making our lives more convenient and easier. With enhanced flexibility and advanced communication methods, the telecom industry gains more customers and creates new revenue streams. According to Grand View Research, the global telecom market size would expand at a compound annual growth rate (CAGR) of 5.4% between 2021-2028. With the rapidly growing digital connectivity, the communication service providers (CSPs) have to deal with large datasets. Datasets that can allow them better to understand their customers, competitors, industry trends and derive valuable insights for decision making.

Read More

Spotlight

High Tech Engineering Center

HTEC (High Tech Engineering Center) is a center of innovation and software development. At HTEC, award-winning creativity sits alongside deep engineering skill. Founded in September 2008 and headquartered in San Francisco, HTEC is a global company with development centers comprising over 300 engineers across Serbia as well as local offices in the United Kingdom, Sweden and China.

Events