“Give me the positions and velocities of all the particles in the universe, and I will predict the future.” – Marquis Pierre Simon de Laplace

Today I read a Bloomberg piece regarding the impending US retail apocalypse. Content aside, I am fascinated by the richness of data presented in the article. Here are a few examples:

Source: America’s ‘Retail Apocalypse’ Is Really Just Beginning

Those are very granular visualization of data down to even county level. No wonder Michael Bloomberg became very, very rich because his company provide data that is very valuable for investors.

Can we do this in Indonesia, down to kecamatan or even kelurahan level?

We will need:

• The means to obtain the data
• Storing the data
• Processing the data
• Using the data for hypothesis testing
• Synthesizing a theoretical body of knowledge from various insights generated from the tested hypotheses.

I recall that once I randomly stumbled to a BukaTalks video in Youtube featuring Crystal Widjaja talking about data science in Gojek. I expected to see a lot of machine learning stuffs, but most of her talk is about how to create an infrastructure that can withstand the tsunami of data Gojek receives every minute and prepare millions of those data to be usable for other team. So much emphasis on SQL, flow of data, server capacity, etc. I was not seeing any machine learning stuffs on the slide. I think that implies that obtaining the data is a far, far greater challenge than whatever that lies downstream of the pipeline. While you just need several very smart people to process and extract insights from the data, obtaining the data is a logistical challenge.

US deep state and private sectors are functioning really well, as demonstrated by the granularity of the data obtained in the figures above. The problem is how to do that in Indonesia. Logistics of obtaining the data is expensive, and utilizing public fund is the only way it makes sense. However, utilizing discretionary public funding and ensuring operational feasibility in the field might be a stretch for Indonesian government. Thus I want to explore whether such effort is feasible to be done by a private company.

As long as the data can be monetized, and being done in a path that ensures profitability from the get go (or at least not burning too much money unsustainably before reaching profitability), I think it should be tried. This firm can be an underwing of a conglomerate, or it can be an independent one. I think US data firms even in the 60s or 70s are independent firms selling data to big companies.

But if the infrastructure has been established, predicting the future is only an inch away.