Replacing Boto3 for Fun and Profit Python is the de-facto language for data science because of its ease of use and performance. But performance comes only because libraries like NumPy offload computation-heavy functions, like matrix multiplication, to optimized C code. Data science tooling and workflows continue to improve, datasets get…

Comparing Python, Go, and Rust Recently a FlashBlade customer had a challenge with listing all custom metadata on objects in a large bucket. A standard S3 LIST request does not return custom metadata, therefore the task requires also issuing a HEAD request for each object. With millions of objects in…

20x Improvement Loading CSV from FlashBlade S3 Pandas is a powerful tool for data exploration and analysis, leveraging Python’s ease of use with the optimized execution of numpy’s arithmetic libraries. But there are three common shortcomings of Pandas: 1) slow IO from object storage, 2) single-threaded execution, and 3) requirement…

Simplify Elastic Operations without Compromising Data Searchability using FlashBlade Scaling log analytics to petabyte scale is hard. Regardless of tool, deploying applications larger than their original design inevitably brings scaling complexity too. At the infrastructure level, the traditional direct-attached storage model (e.g., Hadoop) means that growing capacity requires higher node counts and therefore growing complexity. Each new node brings…

Cloud-native applications must often co-exist with legacy applications. Those legacy applications are hardened and just work, so rewriting can seem hardly worth the trouble. For legacy applications to take advantage of new technology requires bridges, and fuse clients for object storage are a bridge that allow most (but not all)…

Kafka and Elasticsearch Pipelines Made Easier with Kubernetes and Object Storage Collecting and indexing logs from servers, applications, and devices enables crucial visibility into running systems. A log analytics pipeline allows teams to debug and troubleshoot issues, track historical trends, or investigate security incidents. The most commonly deployed pipeline combines…

A Quick and Easy Tool with No Dependencies Spent a few hours trying to debug why Apache Spark on FlashBlade is slower than expected only to realize you have an underlying networking issue? Flashblade-plumbing is a tool to validate NFS and S3 read/write performance from a single client to a…

RESTful interfaces make interacting with modern applications easier for other applications. “Easy” means you can use a variety of languages and tools to build scripts and automation around the application and that REST API, instead of relying only on manual clicks and a GUI. You may already be familiar with…

Dremio is a rapidly growing scale-out analytics engine that is part of a new generation of data lake services, emphasizing the power and flexibility of disaggregated storage like FlashBlade. …