AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |
Back to Blog
Databricks iceberg2/29/2024 "Using Snowflake for data lakes is still new," Derrick Wood, the managing director at Cowen Inc., said. But data lakes - repositories capable of storing massive troves of data in its native format, whether structured or unstructured - have grown increasingly popular, forcing Snowflake to invest in a relevant technology if it wants to court machine-learning experts and data scientists. While Databricks has been selectively making some of its technology available to open source, Snowflake's has been exclusively proprietary. "You're gonna see it emerge as the de facto standard." "It also has the backing of Apple, Netflix, and Amazon is leaning on it," he said. It chose Iceberg because it was far ahead of the alternatives, including both Hudi and Delta Lake, Christian Kleinerman - the chief product officer at Snowflake - said at the company's most recent investor day. Snowflake said at its annual conference earlier this month that it would begin deploying Apache Iceberg, an increasingly popular open-source framework for managing big data. Databricks developed Delta Lake in parallel, announcing its availability in 2019. Netflix originally developed Iceberg before making it available through open source in 2018. "I'm surprised to some extent that Snowflake's bet is on open-source technology, but I think they've identified what it is: probably the most interesting open-source non-Databricks controlled data-lake technology." How Snowflake and Databricks are fighting over the data lake with Iceberg and Delta Lakeīoth Delta Lake and Iceberg try to tackle a problem Databricks and Snowflake are quickly converging on: managing colossal sets of files in an orderly manner to run machine-learning tasks. "They are clearly heading on a collision course or putting themselves on a collision course," Sakib Dadi, the vice president of Bessemer Venture Partners, said. By backing competing open-source frameworks, Databricks and Snowflake are sparking a new phase in the rivalry reminiscent of many earlier open-source rivalries, like Hortonworks and Cloudera. Have an account? Log in.ĭatabricks and Snowflake have increasingly found themselves competing on each others' home turfs, though they still specialize in different venues: data warehousing for Snowflake and machine learning for Databricks. This story is available exclusively to Business InsiderĪnd start reading now. Account icon An icon in the shape of a person's head and shoulders.
0 Comments
Read More
Leave a Reply. |