This video talks about the basic Small Big Data handling techniques for money, compression, batching, and indexing. In particular, you will learn how to apply these methods to NumPy and Pandas. These key concepts can also be applied to some libraries and other data specifics ready to use in your own Python software.
Programming can be a tedious process and it can also be a demanding one. Aside from getting the right code, the right software, there are other equally important factors to consider. In this video, we will discover some useful techniques that you can apply when your data is too big to fit in the memory. What will you do if you have too much data for your project? Chances are that the program will crash which will drastically affect most of your productivity.
According to Itamas Turner-Trauring, one of the ideal solutions is the big data cluster which requires you to have a cluster of computers instead of a typical single computer. The video will also highlight various ways to compress memory including the use of NumPy, Sparse Arrays, and Pandas. You will also learn more about chunking and indexing using the aforementioned arrays. If you want to know more on how to process your data simply and quickly, feel free to watch this learn Python video below.
