What is big data? Introducing the meaning, types, and usage examples for companies

Table of contents of this article

  1. What is big data?
  2. Nature of big data
  3. Types of big data that can be used for business
  4. Big data usage examples
  5. Challenges towards utilizing big data
  6. How to acquire the AI ​​technical skills essential for big data

We are starting to hear more and more cases of people using big data and actually achieving results. There are many companies who want to incorporate big data into their business. On the other hand, some people may remain unclear about what big data is and what it can accomplish.

In this article, we will explain the basic meaning of big data, and then introduce the types of data that can be used in business and examples of their use.

What is big data?

It is generally correct to think of the term big data as referring to “a huge collection of data.” When storing huge amounts of data collected from all over the world, the size can reach terabytes or petabytes.

However, what is important when defining big data is whether it can serve a purpose rather than whether it is large or not. In the context of actual corporate activities, the following data is called “big data.”

– The size is beyond the range that can be handled by general computers and databases
– You can gain some kind of knowledge that is useful for business.

These characteristics indicate that big data is a concept closely related to AI and DX. Let’s take a closer look at the relationship between each.

Big data and AI

In order to handle large amounts of data, appropriate digital technology is inevitably required. For example, the following technologies:

– Storage technology that can store huge amounts of data
– Analysis technology that can process data at high speed

Big data has become practical due to advances in these technologies. Among these, analysis technology using AI has made remarkable progress in recent years. Through advanced analysis, it is now possible to decipher patterns that are difficult to derive from data using human power alone.

Big data and DX

DX is an initiative that utilizes digital technology and data to benefit business. Analysis results backed by data make it possible to select more effective business measures without relying solely on human intuition and experience.

One method that has become more prominent in recent years is the use of big data together with AI. These two are now essential elements of DX, so much so that it can be interpreted as “utilizing AI and big data.”

Nature of big data

Nature of big data

Big data has three properties that make it different from traditional data. We will explain these characteristics of big data, which are called the “three Vs” by their initials.

– Large amount of data (Volume)
– Many types of data (Variety)
– High frequency of data updates (Velocity)

Large amount of data (Volume)

The first “V” is “Volume”, which means “capacity”. This shows that big data has the characteristic of “handling large amounts of data.”

For example, the huge amount of text posted on SNS from all over the world can clearly be said to be a large amount of data. In addition, there is no problem in calling the following data, which is not that huge in practical terms, as big data.

– Business data collected from branches nationwide
– Website access logs accumulated over a long period of time

This property (Volume) of big data is somewhat ambiguous because it does not specify a specific size, but it is also a necessary element to realize the remaining two “Vs” that will be explained later.

There are many types of data (Variety)

The second “V” is “Variety”, which stands for “diversity”. This shows the characteristic of big data that it “includes various types of data.”

For example, the data stored in a typical database is “structured data” that has a predetermined format. In contrast, big data often deals with “unstructured data” whose structure cannot be uniformly determined. Additionally, if you try to collect more data, you will likely have to collect it from a wide variety of sources.

Such diverse data often contains unexpected information. Comprehensive analysis using AI therefore expands the possibility of discovering new knowledge.

Data update frequency is high (Velocity)

The third “V” is “Velocity”, which means “speed”. This shows that big data has the characteristic that new data is added frequently.

For example, in a case where POS data from a nationwide chain store is immediately sent to a data center, the data could increase at a rate of several thousand items per second depending on the time of day. This means that speed is also required when processing data.

With the right use of AI, this data can be analyzed at near real-time speeds. This will make it easier to make business decisions that are highly responsive to changing circumstances.

Types of big data that can be used for business

Types of big data that can be used for business

The 2017 edition of the Information and Communications White Paper published by the Ministry of Internal Affairs and Communications states the following about big data:

The utilization of big data holds the key to realizing data-driven economic growth and social change. The means to collect big data is IoT (Internet of Things), and the means to analyze and utilize big data is AI (artificial intelligence).

The content described here is not all about IoT and AI. However, it can be said that it clearly expresses what is actually happening in many companies that are part of the DX trend. What is important is where to collect big data, which has the characteristics of the “three Vs,” and “for what purpose to analyze it.”

From here, we will explain specifically what types of data have the characteristics of the “three Vs” and can become big data that can be used in business.

– Data collected from an unspecified number of people
– Data collected from users
– Data collected internally
– Various open data

Data collected from an unspecified number of people

The huge amount of data collected from an unspecified number of people through the Internet can definitely be called big data. For example, the following data.

– Connections between users through text posted on SNS, comments, follows, etc.
– Content distributed on video sites
– Access logs of your own website accumulated over a long period of time

These are data that can be widely analyzed and used as clues to understand user needs and the reputation of your company’s products.

Data collected from users

Information obtained from customers of your own services can also become valuable big data if the data size is sufficient.

– Data related to purchasing behavior such as purchase history
– Responses to surveys and user support records
– Location information obtained through smartphones, etc.

These are data that can be used to gain insights that directly lead to improving services and increasing customer satisfaction.

Data collected internally

Various types of data generated from internal activities, such as the following, accumulate and grow on a daily basis, forming big data.

– Various documents created by employees
– Business data and research data accumulated in internal systems
– Sensor data from production lines, etc.

These can be used to improve business processes, such as increasing productivity and safety.

Various open data

Data made publicly available by governments and local governments can also be used as big data. Examples of data include:

– Statistical information such as population
– Weather data
– Hazard maps and disaster information

In addition to these, you may also have access to shared data built through partnerships with partner companies and industry collaborations.

Big data usage examples

Big data usage examples

Big data is already being put into practical use in various industries. Here, we will introduce some examples of use in the following industries.

– Retail: Optimize ordering by predicting demand
– Agriculture: Ensure a stable supply of grapes for brewing
– Medical: Shorten the research and development period for new drugs
– Communication: Predict car congestion and make it available to the surrounding area
– Tourism: Behavioral analysis to revitalize the hot spring town
– Sports: Digital commentators predict the next pitch
– Government: AI supports issuing evacuation orders during disasters

Retail: Optimize ordering with demand forecasting

Labor shortages and food waste are major issues in the retail industry, such as supermarkets and convenience stores. If you rely on the intuition of veteran employees to order products, there is a risk that you will not be able to properly maintain inventory if that employee takes a leave of absence or leaves the company.

Ito-Yokado, a member of the Seven & i Group, has adopted AI technology to predict daily demand for perishable foods. Its unique feature is that it improves prediction accuracy by combining different types of data, such as weather forecasts and event information.

The details of this case are also introduced in this article , so please refer to it as well.

Agriculture: Stable supply of grapes for brewing

In the field of agriculture, stable production of crops has become difficult due to abnormal weather in recent years. The reality is that even if we look at weather data, we don’t know what to do.

In Takayama Village, Nagano Prefecture, efforts were made to formalize the know-how of cultivating grapes for winemaking, which had previously been a matter of individual skill. By collecting a variety of data from sensors inside the farm, the aim is to clarify the conditions under which diseases occur. We have also introduced notifications from bots, allowing us to take necessary measures in a timely manner.

The details of this case are also introduced in this article , so please refer to it as well.

Medical: Shorten the research and development period for new drugs

In recent years, the target of new drug development has shifted to intractable diseases with fewer cases. This makes research and development increasingly difficult, and the challenge is how to shorten the development period.

Therefore, pharmaceutical companies are working to improve the efficiency of research and development by converting various compounds that can be used as drug ingredients into data. This is a system that uses AI to pick up optimal combination candidates from a huge amount of data.

The details of this case are also introduced in this article , so please refer to it as well.

Communication: Predicting car traffic congestion and disclosing it to surrounding areas

Communication services require different quality depending on their purpose. The automotive industry requires highly real-time communication for autonomous driving and traffic information.

NEXCO East is releasing AI-based traffic congestion predictions for users of the Tokyo Bay Aqua Line. By using NTT Docomo’s communication technology and real-time demographic statistics, we have made highly accurate predictions possible.

The details of this case are also introduced in this article , so please refer to it as well.

Tourism: Revitalizing hot spring towns with behavioral analysis

Revitalizing hot spring towns with behavioral analysis

Kinosaki Onsen, located in northern Hyogo Prefecture, has a service that allows you to check in to inns and tour hot springs without having to use your wallet, all you need is a smartphone. By quantitatively analyzing usage history, it is now possible to predict people’s behavior based on data, which previously could only be understood qualitatively.

What can be obtained from this data is information that is familiar to tourists, such as group composition, excursion routes, and usage times for each facility. This information is used to understand and improve the effects of events and advertisements held by hot spring towns.

Sports: Digital commentator predicts next pitch

AI can sometimes present analysis results that cannot be arrived at with human thinking.

ZUNO, a digital sports commentator developed by Dentsu Inc., is a system that can predict pitching according to the situation of professional baseball games. By analyzing data from over 3 million past at-bats, we are now able to understand player trends that human commentators have not noticed before. When this system was used to predict pitching in an actual game, it achieved a higher accuracy rate than humans.

Government: AI supports issuing evacuation orders during disasters

There are also examples of efforts to utilize AI in evacuation orders issued by local governments during disasters.

Spectee Co., Ltd. has developed the crisis management solution “Spectee Pro” in a demonstration project in Nagoya City. AI analyzes river water levels, precipitation, information from SNS, etc. in real time. Predictions based on data enable decisions that do not rely solely on the person in charge’s rules of thumb. This means that we will continue to create a system that can issue accurate evacuation information without delay.

Challenges towards utilizing big data

Challenges towards utilizing big data

There are also challenges in utilizing big data. We will explain common issues that companies that are thinking of actively utilizing big data should consider.

– Data management with consideration for personal information is required
– The quality of data determines the accuracy of analysis
– Human resources who can bridge data and business are required

Data management that takes personal information into consideration is required.

Big data often includes personal data such as customer information. In addition to data that can directly identify individuals, such as names and contact information, fingerprints and DNA are also personal data that must be handled with care. When using personal data, measures such as “obtaining consent from the provider before use” or “anonymizing the data and using it only under certain conditions” are required.

Data must also be managed and operated strictly. It will be necessary to improve traceability and be able to understand operations that involve copying or moving data. In addition, data protection is also an issue. It is important to strengthen system security and prevent unintentional data leakage.

Data quality determines analysis accuracy

Data quality is important for highly accurate analysis. However, the quality of the data that is simply collected is likely to vary. For example, there may be typos, garbled characters, duplicates, or missing data when entering data.

Additionally, data collected across departments often causes problems when collated. If there are “variations” in the notation or if IDs and identification numbers are not unified, there are cases where it is not possible to link data that should originally be related.

In order to obtain satisfactory analysis results, this data requires a process called “preprocessing.” Correct any duplication or missing data in advance, and align the format to make it easier to analyze.

We need people who can bridge data and business.

For companies to gain meaningful insights from their data, they must conduct analytics that meet business demands. However, the reality is that many of the skills needed to utilize big data are highly specialized.

For example, in order to make the most of data, a data analyst’s ability to identify the appropriate analysis method from among many is essential. Engineers are also required to build AI models and implement them into actual systems.

Companies that want to effectively utilize data for business must think about how to secure these human resources. To this end, options include building partnerships with external vendors and developing talent in-house.

How to acquire the AI ​​technical skills essential for big data

Big data is a source of information that brings new insights to your business. For effective use, it can be said that combination with AI is essential.

On the other hand, AI is a technology that requires a high level of expertise. If you are concerned about the lack of human resources who can properly handle AI, please consider using our ” AI Implementation Support ” service. Experienced AI engineers support speedy development and technology acquisition.

Leave a Comment