Writing to make sense of HPC, storage, and system design
Last month was the 2026 ISC High Performance Conference in the beautiful (and sweltering) Hamburg, Germany. It was my fifth time attending in-person, and it has fast become my favorite HPC community conference of the year. Some combination of the program, the people, and the venue that strikes the right balance of technology, community, and commerce that always leaves me coming away with a enough…
I attended the 2026 Salishan Conference on High Speed Computing last month, and it was a week well spent in coastal Oregon hearing what many of the world's experts in scientific computing are worried about. Although I took copious notes and many of the presentations are online now , I'm not really sure how closed-door the week is meant to be. As such, I don't feel quite comfortable sharing a…
I recently attended GTC26, NVIDIA's flagship annual conference, and it was a blazing hot week full of equal parts glitz and technology. And when I say blazing hot, I mean that literally--San Jose was in the midst of a record-breaking heat wave with temperatures consistently above 30C every day. Despite the sizzle though, the steak came out like it got microwaved: hot in a few places, but cold in…
Dan Reed recently published an essay, HPC In An AI World , that summarizes a longer-form statement piece he co-authored with Jack Dongarra and Dennis Gannon called Ride the Wave, Build the Future: Scientific Computing in an AI World . It's worth a read since, as with much of Dr. Reed's writing, it takes a necessary, hard look at where the HPC community needs to look as the world underneath it…
The annual SC conference was held last week, drawing over 16,000 registrants and 560 exhibitors to St. Louis, Missouri to talk about high-performance computing, artificial intelligence, infrastructure, and science. It was my tenth time attending in-person (12th overall), and as is always the case, it was a great week to reconnect with colleagues, hear what people are worrying about, and get a…
I recently decided to leave Microsoft after having spent just over three years there, first as a storage product manager, then as a compute engineer. Although I touched many parts of Azure's infrastructure during that time, everything I did was at the intersection of large-scale supercomputing and hyperscale cloud. There was no shortage of interesting systems to figure out and problems to solve,…
I had the pleasure of attending the 40th annual ISC High Performance conference this month in Hamburg, Germany. It was a delightful way to take the pulse of the high-performance computing community and hear what the top minds in the field are thinking about. The main foyer of Congress Center Hamburg, and the view that greeted me on the first morning of ISC'25. The conference felt a little quieter…
I attended GTC for the first time this year and got to experience what turned out to be one of the strangest, over-the-top conferences I've ever attended. It took many typical conference activities (technical talks, an exhibit floor, after-hours parties) and mashed them together with some distinctively unusual conference elements like a keynote in a sports arena, a night market, and a live music…
The illustrious Jeff Denworth recently posted a hot take across social media, claiming that training large language models (LLMs) doesn't require massive, expensive parallel file systems: As someone who's been working on one of the largest supercomputers on the planet --one that has no parallel file system at all--I was surprised by how many incredulous or curious responses followed. I guess…
The premiere annual conference of the high-performance computing community, SC24, was held in Atlanta last week, and it attracted a record-shattering number of attendees-- nearly 18,000 registrants , up 28% from last year! The conference felt big as well, and there seemed to be a lot more running between sessions, meetings, and the exhibition floor. Despite its objectively bigger size though, the…
FASST is an initiative within the US Department of Energy to build and deploy the world's most powerful AI systems for science and is shaping up to be the next big thing after the US Exascale Computing Project. With an eye towards an initial $12 billion budget , the program represents a bold vision of positioning the US government as the leader in AI for science. However, unlike exascale computing…
As I spend more time in the AI infrastructure business, I've been thinking more and more about the government's role in AI . There's no shortage of opinions and position papers on the topic, and sadly, most of them are written from the perspective of the government rather than the AI industry. As a result, they are often full of misunderstandings, misleading statements, or ideas about the world…
June 2024 marked two years since I left my job at one of the world's most prestigious government HPC centers for a job in one of the world's largest technology corporations . In that time, the world of HPC has changed dramatically; just six months after I started, ChatGPT was released and triggered a gold rush in AI that is now overshadowing traditional scientific computing. This shift brought…
I had the great pleasure of attending the ISC High Performance conference this month, marking the fifth time I've attended what has become one of my top must-attend industry conferences of the year. This year was particularly meaningful to me because it is the first time that: I attended ISC as a Microsoft employee. This is also the first time I've attended any HPC conference since I changed my…
I have the great fortune of being able to attend to the 2024 ISC High Performance conference in Hamburg next week where I will be both speaking and listening throughout the contributed program and the workshop day. I'm going in with a few goals in mind: Connect with the international HPC community Engaging with the HPC community--whether reconnecting with former collaborators and friends, talking…
A paper titled " Optimizing Distributed Training on Frontier for Large Language Models " has been making its rounds over the last few weeks with sensational taglines saying the authors trained a trillion-parameter model using only a fraction of the Frontier supercomputer . The superficiality of the discourse around this paper seemed suspicious to me, so in the interests of embracing my new job in…
The largest high-performance computing industry conference of the year, SC23, was held in Denver last week. This year's conference attracted over 14,000 attendees and 438 exhibitors , finally breaking pre-pandemic records, and it solidly felt like the old days of the conference in terms of breadth of attendees, the technical program, and overall engagement and interaction across the community.…
The biggest annual conference in HPC, the SC conference , was recently held in Dallas, Texas in its second hybrid incarnation since being all-remote for the pandemic. This year attracted over 11,000 attendees which is much closer to the pre-pandemic high of 14,000 than last year's 7,000, and judging from the crushed conference rooms and busy expo floor, it looks like SC is not that much worse for…
When word started to spread that I was leaving my job at NERSC for Microsoft, a lot of people either directly or indirectly attributed my decision to being one motivated by money. Rationalizing my decision to leave is certainly a lot easier with this "Glenn was lured away with bags of cash" narrative, but that wasn't really a factor when I chose to move on. Rather, my decision is a reflection of…
This post is a long-form dump of some thoughts I've had while testing all-flash file systems this past year, and bits of this appear in a presentation and paper I'm presenting at PDSW'21 about new benchmarking techniques for testing all-flash file systems. "How many IOPS do you need?" I'm often asked this by storage vendors, and the question drives me a little bonkers. I assume they ask it because…
The HPC industry's biggest conference, SC , was held virtually over the last two weeks. Although the original plan to hold it in Atlanta was supplanted by all-virtual format, it still managed to be a whirlwind show full of product showcases, research presentations, and interesting talks, panels, and workshops. The virtual format certainly wasn't the same as attending in-person, but some of the…
This year was the first all-virtual Parallel Data Systems Workshop , and despite the challenging constraints imposed by the pandemic, it was remarkably engaging. The program itself was contracted relative to past years and only had time for three Work-In-Progress (WIP) presentations, so it was a little difficult to pluck out high-level research trends and themes. However, this year's program did…
When my employer announced that we were all going to work remotely back in March, I had no semblance of a home office and had to scramble to figure out how to set up a space in my small urban apartment that would be suitable for days dominated by videoconferencing. Thinking it'd be just a few weeks, I thought I could ride it out using my guest bedroom's writing desk, but by the time June rolled…
The delivery of Japan's all-CPU Fugaku machine and the disclosure of the UK's all-CPU ARCHER 2 system amidst the news, solidly "pre-Exascale" machines with pre-exascale budgets, is opening old wounds around the merits of deploying all-CPU systems in the context of leadership HPC. Whether a supercomputer can truly be "leadership" if it is addressing the needs of today using power-inefficient,…
Although I've known a lot of the parameters and features surrounding ZFS since its relative early days, I never really understood why ZFS had the quirks that it had. ZFS is coming to the forefront of HPC these days though--for example, the first exabyte file system will use ZFS --so a few years ago I spent two days at the OpenZFS Developer Summit in San Francisco learning how ZFS works under the…