Comparing Python, Go, and Rust Recently a FlashBlade customer had a challenge with listing all custom metadata on objects in a large bucket. A standard S3 LIST request does not return custom metadata, therefore the task requires also issuing a HEAD request for each object. With millions of objects in…

20x Improvement Loading CSV from FlashBlade S3 Pandas is a powerful tool for data exploration and analysis, leveraging Python’s ease of use with the optimized execution of numpy’s arithmetic libraries. …

Simplify Elastic Operations without Compromising Data Searchability using FlashBlade Scaling log analytics to petabyte scale is hard. Regardless of tool, deploying applications larger than their original design inevitably brings scaling complexity too. At the infrastructure level, the traditional direct-attached storage model (e.g., Hadoop) means that growing capacity requires higher node counts and therefore growing complexity. Each new node brings…

Cloud-native applications must often co-exist with legacy applications. Those legacy applications are hardened and just work, so rewriting can seem hardly worth the trouble. For legacy applications to take advantage of new technology requires bridges, and fuse clients for object storage are a bridge that allow most (but not all)…

Kafka and Elasticsearch Pipelines Made Easier with Kubernetes and Object Storage Collecting and indexing logs from servers, applications, and devices enables crucial visibility into running systems. A log analytics pipeline allows teams to debug and troubleshoot issues, track historical trends, or investigate security incidents. The most commonly deployed pipeline combines…

A Quick and Easy Tool with No Dependencies Spent a few hours trying to debug why Apache Spark on FlashBlade is slower than expected only to realize you have an underlying networking issue? Flashblade-plumbing is a tool to validate NFS and S3 read/write performance from a single client to a…

RESTful interfaces make interacting with modern applications easier for other applications. “Easy” means you can use a variety of languages and tools to build scripts and automation around the application and that REST API, instead of relying only on manual clicks and a GUI. You may already be familiar with…

Dremio is a rapidly growing scale-out analytics engine that is part of a new generation of data lake services, emphasizing the power and flexibility of disaggregated storage like FlashBlade. …

In this post, I look at what it takes to list all keys in a single bucket with 67 billion objects and build a simple list benchmark program in Golang. …