Episode 34 - Data Engineering with Joe Crobak, Foundations of Smart Software
Now with extra mappers and reducers!
I talk to Joe Crobak of the Data Engineering Weekly newsletter about data engineering, and it's importance in modern software projects. We cover:
- The definition of data engineering
- Hadoop, MapReduce, and Spark
- How data engineering has evolved over the last decade.
- Joe's experiences working in government vs. startups. The US Digital service, and the paperwork reduction act.
Guest
Joe Crobak is software developer that has worked on several data-powered consumer products at Foursquare and other startups in NYC. He worked for two years in the federal government as a founding member of the United States Digital Service team at the Department of Health and Human Services. Joe's Data Eng Weekly newsletter, which covers distributed systems and big data, reaches over 11,000 subscribers. He lives in Washington, DC with his wife and two dogs.
Joe's site at dataengweekly.com
Joe’s Twitter
Joe’s Medium - check out The 10 most Surprising Things I Learned Working in the Government
Previous Episodes
Episode 13 on Applied Artificial Intelligence, where we also looked at organizational hurdles to creating AI product
Book Recommendations
Winning with Data - a business book about big data and infrastructure
Hadoop: The definitive Guide
