From the course: Google Cloud Professional Data Engineer Cert Prep: 2 Building and Operationalizing Data Processing Systems

Unlock this course with a free trial

Join today to access over 24,100 courses taught by industry experts.

The challenges of big data

The challenges of big data - Google Cloud Tutorial

From the course: Google Cloud Professional Data Engineer Cert Prep: 2 Building and Operationalizing Data Processing Systems

Start my 1-month free trial Buy for my team

The challenges of big data

“

- [Instructor] One of the more interesting ways of dealing with data is to consider the concept of three Vs. The volume of data, the variety of data and the velocity of data. So let's take a look at the volume. So this could be an easy one right off the top of your head if it's terabytes or petabytes of data, right? It's big data and it's going to cost some challenges. Like for example, you can't put petabytes of data on your laptop. There's no storage for it. You'll have to use a cloud-based system or some, you know file server to manage that data. And that's one of the challenges that the cloud does actually help you solve. And even if you have your own data center you can do a hybrid cloud solution. Now, another variety problem is the fact that you have CSV files, SQL files, binary files, API files, you know, key value files, video files all of these varieties can cause an issue and you have to actually figure out how…

Contents