Problem Solving with Data

For those just getting started with advanced scientific computing techniques, here are four steps to efficiently turn data into decisions with business value.

Author: Ryan Swindeman, Scientific Software Developer

In the 4 minute video below, Enthought scientist Ryan Swindeman puts data into context as foundational to any digital transformation initiative, setting out four fundamental steps for data to science problems.

1. Data Preparation (or Data Conditioning): This is the essential, first step in a digital project. Data must be clean and accessible. Access to data must be quick, and reliable. The data must be cataloged or categorized, so that there is consistency in how it is reached and integrated into projects. Data preparation must be in service of addressing a business need or objective, to solve a specific problem, and not be a case of ‘we need to organize our data’.

2. Data Visualization: Visualizing data is important as a starting point to understanding a problem. This involves looking at the data in its native domain, identifying trends, and from there possibly transforming it to a different domain, cross-plotting to look for relationships, or running statistics as a way to discover features. Visualization is also a reliable way to increase efficiency in problem-solving. The understanding gained through visualization is essential for deep learning – if you do not understand the underlying trends or relationships in the data, you will not understand the outcomes produced by any AI/ML/Deep Learning.

3. Modeling and Optimization: This step uses the underlying dynamics or physics of the problem, and the applications are endless. (In geophysics, this is often called forward modeling and inversion.) Most critically, modeling and optimization allows scientists to prove (or disprove) hypotheses very quickly, enabling teams to test, iterate and change strategy, often resulting in problems being solved quickly.

4. AI/ML/Deep Learning: These advanced computing techniques are related, and differ in important ways. Unlike modeling and optimization, or inversion (which is a physics-based approach), AI/ML/Deep Learning is a data-driven approach. These techniques are beneficial if forward modeling and optimization are not possible because of a lack of understanding of the underlying physics, or if the physics leads to too many approximations. The problem-solving and analytical power of AI/ML/Deep Learning becomes obvious in pattern recognition or texture analysis.

These four steps provide a robust sequence for solving problems using data, whether a small set or large, fundamental to digital transformation projects.

 

About the Author

Ryan Swindeman, Scientific Software Developer, holds a M.S. in geophysics from the University of Texas at Austin and a B.S. in physics from the University of Illinois at Urbana-Champaign, with graduate research in computational seismology.

Share this article:

Related Content

True DX in the Pharma R&D Lab Defined by Enthought

Enthought’s team in Japan exhibited at the Pharma IT & Digital Health Expo 2022 life sciences conference in Tokyo, to meet with pharmaceutical industry leaders…

Read More

Life Sciences Labs Optimize with New Digital Technologies and Upskilling

Labs are resetting the trajectory for drug development: reducing timelines from years to months; decreasing costs from billions to millions; and gaining an advantage by…

Read More

Webinar Q&A: Accelerating Product Reformulation with Machine Learning

In our recent C&EN Webinar: Accelerating Consumer Products Reformulation with Machine Learning, we demonstrated how to leverage digital tools and technology to bring new products…

Read More

Scientists Who Code

Digital skills personas for success in digital transformation The digital skills mix varies widely across companies, from those just starting to invest in digital transformation…

Read More

The Challenges of Scaling Digital Advances in Life Sciences

Scaling innovations in R&D can be considered challenging in two dimensions: from lab to lab; and from the lab to the end customer. A 100x…

Read More

Digital Transformation in Practice

Taken from the webinar, the frequent strategy of digitalization-layering digital tools and technology onto existing processes-provides incremental value, which soon flattens out. At the other…

Read More

Giving Visibility to Renewable Energy

The ultimate project goal of EnergizAIR Infrastructure was to raise individual awareness of the contribution of renewable energy sources, and ultimately change behaviors. Now ten…

Read More

Machine Learning in Materials Science

The process of materials discovery is complex and iterative, requiring a level of expertise to be done effectively. Materials workflows that require human judgement present…

Read More

FORGE-ing Ahead: Charting the Future of Geothermal Energy

A microseismic event loaded from the Frontier Observatory for Research in Geothermal Energy (FORGE) distributed acoustic sensing (DAS) data into a Jupyter notebook showing energy…

Read More

Strategy, Digitalization and Global Trends – C Suite Reflections

Key points are presented from the first of a series of LinkedIn articles where JSR Board Chairman Mitsunobu Koshiba provides thought provoking insights on business…

Read More

Join Our Mailing List!

Sign up below to receive email updates including the latest news, insights, and case studies from our team.