Got Technology, Need Data? Here's Where to Get It...For Free

| November 30, 2016

article image
Have you ever wanted to assess or explore a new BI, analytics, data visualization, data science, etc. technology, but struggled to find a data set to use for your assessment and exploration activities? Have you grown tired of analyzing the “typical” datasets–think census, stock market, and employment and inflation rate data–known to be publicly available? Are you seeking data relevant to a particular concept, but aren’t sure how to collect it? Are you wanting to generate world-changing insights on a fresh, rarely analyzed, concept?In the data-driven, data science and analytics obsessed world we live in today, obtaining robust data sets to fill each of these use cases is fortunately not the monumental task it was just a few years ago. From static historical government data to real-time social media streams to private company data released for public consumption, the internet is now your data oyster.

Spotlight

BuilDATAnalytics

BuilDatAnalytics is a business intelligence company for the commercial construction industry. BDA provides field crew with its flagship, patent pending software platform called CTBIM. CTBIM captures real-time field activities, which enables the production of insightful analytics for project owners and contractors. Simply stated, CTBIM™ transforms the current way of capturing and managing information during pre-construction, construction and post-construction to an integrated, streamlined and more efficient process. Users of CTBIM™ realize fewer costly errors, a reduction of risk, more efficient use of time and greater profits.

OTHER ARTICLES

Data Analytics the Force Behind the IoT Evolution

Article | April 3, 2020

Primarily,the IoT stack is going beyond merely ingesting data to data analytics and management, with a focus on real-time analysis and autonomous AI capacities. Enterprises are finding more advanced ways to apply IoT for better and more profitable outcomes. IoT platforms have evolved to use standard open-source protocols and components. Now enterprises are primarily focusing on resolving business problems such as predictive maintenance or usage of smart devices to streamline business operations.Platforms focus on similar things, but early attempts at the creation of highly discrete solutions around specific use cases in place of broad platforms, have been successful. That means more vendors offer more choices for customers, to broaden the chances for success. Clearly, IoT platforms actually sit at the heart of value creation in the IoT.

Read More

How big data can help the homeless

Article | March 12, 2020

Homeless policy needs to join the big data revolution. A data tsunami is transforming our world. Ninety percent of existing data was created in the last two years, and Silicon Valley is leveraging it with powerful analytics to create self-driving cars and to revolutionize business decision-making in ways that drive innovation and efficiency.Unfortunately, this revolution has yet to help the homeless. It is not due to a lack of data. Sacramento alone maintains data on half a million service interactions with more than 65,000 homeless individuals. California is considering integrating the data from its 44 continuums of care to create a richer pool of data. Additionally, researchers are uncovering troves of relevant information in educational and social service databases.These data, however, are only useful if they are aggressively mined for insights, looking for problems to solve and successful practices to replicate. At that juncture California falls short.

Read More

How Machine Learning Can Take Data Science to a Whole New Level

Article | December 21, 2020

Introduction Machine Learning (ML) has taken strides over the past few years, establishing its place in data analytics. In particular, ML has become a cornerstone in data science, alongside data wrangling, and data visualization, among other facets of the field. Yet, we observe many organizations still hesitant when allocating a budget for it in their data pipelines. The data engineer role seems to attract lots of attention, but few companies leverage the machine learning expert/engineer. Could it be that ML can add value to other enterprises too? Let's find out by clarifying certain concepts. What Machine Learning is So that we are all on the same page, let's look at a down-to-earth definition of ML that you can include in a company meeting, a report, or even within an email to a colleague who isn't in this field. Investopedia defines ML as "the concept that a computer program can learn and adapt to new data without human intervention." In other words, if your machine (be it a computer, a smartphone, or even a smart device) can learn on its own, using some specialized software, then it's under the ML umbrella. It's important to note that ML is also a stand-alone field of research, predating most AI systems, even if the two are linked, as we'll see later on. How Machine Learning is different from Statistics It's also important to note that ML is different from Statistics, even if some people like to view the former as an extension of the latter. However, there is a fundamental difference that most people aren't aware of yet. Namely, ML is data-driven while Statistics is, for the most part, model-driven. This statement means that most Stats-based inferences are made by assuming a particular distribution in the data, or the interactions of different variables, and making predictions based on our mathematical models of these distributions. ML may employ distributions in some niche cases, but for the most part, it looks at data as-is, without making any assumptions about it. Machine Learning’s role in data science work Let’s now get to the crux of the matter and explore how ML can be a significant value-add to a data science pipeline. First of all, ML can potentially offer better predictions than most Stats models in terms of accuracy, F1 score, etc. Also, ML can work alongside existing models to form model ensembles that can tackle the problems more effectively. Additionally, if transparency is important to the project stakeholders, there are ML-based options for offering some insight as to what variables are important in the data at hand, for making predictions based on it. Moreover, ML is more parametrized, meaning that you can tweak an ML model more, adapting it to the data you have and ensuring more robustness (i.e., reliability). Finally, you can learn ML without needing a Math degree or any other formal training. The latter, however, may prove useful, if you wish to delve deeper into the topic and develop your own models. This innovation potential is a significant aspect of ML since it's not as easy to develop new models in Stats (unless you are an experienced Statistics researcher) or even in AI. Besides, there are a bunch of various "heuristics" that are part of the ML group of algorithms, facilitating your data science work, regardless of what predictive model you end up using. Machine Learning and AI Many people conflate ML with AI these days. This confusion is partly because many ML models involve artificial neural networks (ANNs) which are the most modern manifestation of AI. Also, many AI systems are employed in ML tasks, so they are referred to as ML systems since AI can be a bit generic as a term. However, not all ML algorithms are AI-related, nor are all AI algorithms under the ML umbrella. This distinction is of import because certain limitations of AI systems (e.g., the need for lots and lots of data) don't apply to most ML models, while AI systems tend to be more time-consuming and resource-heavy than the average ML one. There are several ML algorithms you can use without breaking the bank and derive value from your data through them. Then, if you find that you need something better, in terms of accuracy, you can explore AI-based ones. Keep in mind, however, that some ML models (e.g., Decision Trees, Random Forests, etc.) offer some transparency, while the vast majority of AI ones are black boxes. Learning more about the topic Naturally, it's hard to do this topic justice in a single article. It is so vast that someone can write a book on it! That's what I've done earlier this year, through the Technics Publications publishing house. You can learn more about this topic via this book, which is titled Julia for Machine Learning(Julia is a modern programming language used in data science, among other fields, and it's popular among various technical professionals). Feel free to check it out and explore how you can use ML in your work. Cheers!

Read More

CYBERSECURITY STRATEGIES TO MAKE IT NETWORKS MORE RESILIENT TO CYBERATTACKS

Article | February 28, 2020

The increasing use of advanced technologies and the internet have created an attack surface for malicious attackers. With these progressions, businesses’ IT systems are now more vulnerable which has led them to leverage innovative cybersecurity strategies that can thwart and make their networks more resilient to cyberattacks. Cybercriminals can use a variety of attacks against individuals or businesses like accessing, changing or deleting sensitive data; extracting payment; interfering with business processes and more.These kinds of attacks present an evolving danger to organizations, employees and consumers, and can cost them reputation, finances and personal lives to some extent. So, in order to protect IT networks from cyberattacks, it is significant to be aware of the various aspects of cybersecurity.

Read More

Spotlight

BuilDATAnalytics

BuilDatAnalytics is a business intelligence company for the commercial construction industry. BDA provides field crew with its flagship, patent pending software platform called CTBIM. CTBIM captures real-time field activities, which enables the production of insightful analytics for project owners and contractors. Simply stated, CTBIM™ transforms the current way of capturing and managing information during pre-construction, construction and post-construction to an integrated, streamlined and more efficient process. Users of CTBIM™ realize fewer costly errors, a reduction of risk, more efficient use of time and greater profits.

Events