11 Nov 19

Article that covers the following topics related to computing large datasets without the need to set up a distributed cluster:

  • Why is RAM needed in the first place
  • The easiest way to process data that doesn’t fit in memory: spending some money.
  • The three basic software techniques for handling too much data: compression, chunking, and indexing.
by mlb