9/25/2023 0 Comments Google highfive extension![]() This means it may not be as up-to-date as other components in your stack. One possible issue with Optimus is that it's still under active development but its last official release was in 2020. Optimus also comes bundled with processors for handling common real-world data types like email addresses and URLs. cols() accessors to make it easy to do things like sort a dataframe, filter by column values, alter data according to criteria, or narrow the range of operations based on some criteria. The data manipulation API resembles Pandas, but adds. Data can be loaded in from and saved back out to Arrow, Parquet, Excel, a variety of common database sources, or flat-file formats like CSV and JSON. Optimus can use Pandas, Dask, CUDF (and Dask + CUDF), Vaex, or Spark as its underlying data engine. Optimus is an all-in-one toolset for loading, exploring, cleansing, and writing data back out to a variety of data sources. ![]() One of the least enviable jobs you can be stuck with is cleaning and preparing data for use in a DataFrame-centric project. Querying works like any other SQL-powered relational database, but with additional built-in features like the ability to take random samples of data or construct window functions.ĭuckDB also has a small but useful collection of extensions, including full-text search, Excel import/export, direct connections to SQLite and PostgreSQL, Parquet file export, and support for many common geospatial data formats and types. The resulting databases can also be partitioned into multiple physical files for efficiency, based on keys (e.g., by year and month). And there's no separate software suite to configure you can get it running in a Python environment with a single pip install command.ĭuckDB can directly ingest data in CSV, JSON, or Parquet format. But it gives you all the things you expect from a conventional database, like ACID transactions. Since it runs as an in-process library, rather than a separate application, it's lightweight and responsive.ĭuckDB is a little like someone answered the question, "What if we made SQLite for OLAP?" Like other OLAP database engines, it uses a columnar datastore and is optimized for long-running analytical query workloads. DuckDBĭata science folks who use Python ought to be aware of SQLite-a small, but powerful and speedy, relational database packaged with Python. The results can be funneled into a Pandas or PyArrow DataFrame, or into Modin, Dask, or Polars by way of PyArrow. Data in PostgreSQL, for instance, can be loaded this way by specifying a partition column.Īside from PostgreSQL, ConnectorX also supports reading from MySQL/MariaDB, SQLite, Amazon Redshift, Microsoft SQL Server and Azure SQL, and Oracle. This allows for optimizations like being able to load from a data source in parallel with partitioning. You need the investment and team to do it really well.Like Polars (which I'll discuss soon), ConnectorX uses a Rust library at its core. “This is not a rapid-release, early prototype type of thing it’s not something you can solve on a million dollars. “What gives me confidence is we’re a group of people that has been in this industry for a while, have worked on this before, and have investors who know this,” Sinha said. ![]() Of course, he also talks a big game about the market opportunity: No less than “every business in the world.” Sinha said his focus is on building an enterprise tool, not a consumer tool that’s adapted for companies. ![]() So, what comes after that? “We have devices that know who we are and where we’re supposed to be, and cloud services that are smart about who’s trying to connect with who,” Sinha said. Sinha makes Highfive sound like a magic trick, but he acknowledged that it will include normal earthbound tools like voice and video communication. “We all know that the first 15 minutes of any meeting is spent getting connected,” he said in an interview this week. “Microsoft, Google and Cisco have products in this space, but there’s the same frustration, no matter who you talk to.” It will “reimagine the way we communicate at work,” Sinha said.īut, for now, Sinha doesn’t want to say what exactly Highfive is doing. The company promises a beta launch in two weeks, and broad availability next year. Highfive already has a team of 25, including co-founder Jeremy Roy, also from DocVerse, and others from Apple, Mozilla and Amazon. Sinha has used that cred, along with some yet-to-be-publicly-disclosed ideas about how to create a better sort of videoconferencing and screen-sharing tool for the modern mobile age, to raise $13.5 million for his new startup Highfive from General Catalyst Partners, Andreessen Horowitz, Google Ventures, and cloud founder/CEO rock stars Marc Benioff (Salesforce), Drew Houston (Dropbox) and Aaron Levie (Box).
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |