Today I got an email from AWS Health about my personal account: Our records indicate that one or more applications in this account made AWS API calls using the AWS SDK for Java 1.x within the last 16 days. Hmmm … I think the only thing I did with that account in the last several months was uploading backup files to S3 using the CLI (which is written in Python). Maybe I have a Lambda running…
It's been a little over a year since I bought a CX-70. Overall, I really like it: it's comfortable, handles well, looks great inside and out, has enough cargo space for my needs, and is able to pull my 3500 pound travel trailer with ease. But there are a few things that a really don't like, to the extent that Mazda might not be on my list the next time I look for a car. I've listed them here in…
This is the third time that I'm writing about this topic. The first time was in 2018 , the second in 2021 . In the interim, AWS has announced a steady stream of improvements, most recently (October) increasing read throughput to 60 MB/sec. I wasn't planning to revisit this topic. However, I read Tim Bray's post on the Bonnie disk benchmark, and it had the comment “it’d be fun to run Bonnie…
AWS's recent fait accompli deprecation of CodeCommit and six other services was a shock to me. * Not because CodeCommit was a particularly good product — it wasn't — but because it could have been one with some investment. While there are many Git repository services in the world, CodeCommit was part of the AWS ecosystem, giving it some unique capabilities. Capabilites that AWS never…
AWS just announced a natural language query generator for CloudTrail Lake . Using it, you can write prompts like “What are my top errors in the past month?” and it will spit out the SQL to get the answer. On the surface, that seems great: CloudTrail events can be very complex: they hold the request and response data for thousands of unrelated API operations. In my experience,…
I recently made a minor change to my logging library . And, minor though it was, I run a full suite of integration tests before any release. I wasn't expecting any problems, so was surprised when the first test against Kinesis failed. More puzzling: the primary assertions in the test all passed; the records were properly written to the stream. What failed was an assertion that there weren't any…
I have a “tax PC”: an old HP running Windows 10, that spends ten months of the year in a closet, and two months hooked up to a segment of my home network where it's unlikely to be attacked by the various network-attached devices living in my house. Each February I spend a couple of days applying the year's accumulated Windows updates, then I install Turbo-Tax so that I can do my taxes.…
Tip 26: “select” isn't broken. The OS is probably not broken. And the database is probably just fine. The Pragmatic Programmer Words to live by — although I did once find a problem with Oracle's index management, which manifested in select . In my opinion, it's the maxim that defines a senior developer. Junior developers always look outside themselves whenever they run into a…
Several years ago I wrote about connecting CloudWatch Logs into into a logging pipeline built on Kinesis Streams and Kinesis Firehose. To make that work, I wrote a Lambda . AWS announced EventBridge Pipes at re:Invent 2022, as an easy way to connect messaging services. Connecting CloudWatch Logs to Kinesis seemed like a good way to kick the tires on this service. It still requires a custom Lambda,…
Here in Philadelphia, the last day of 2022 was rainy and foggy; a day to stay inside. That, combined with $100 in AWS credits that were due to expire with the new year, made me decide to add another entry to my series of posts evaluating build performance . This time, I'm comparing my desktop PC to the current AWS king of the hill, an i4i.metal instance. If you don't make a habit of reading EC2…
I just made what I hope is the last release of my log4j-aws-appenders library. It started almost six years ago as a "weekend project" to produce a Log4J 1.x appender that wrote output to CloudWatch Logs. It has expanded over the years: I added Kinesis Streams and SNS as destinations, added support for Log4J 2.x and Logback, added support for the AWS 2.x Java SDK, and maintained the code as…
Chasing a bug can be one of the most frustrating parts of software development. All you have to start with is an observed behavior. Hopefully you can replicate that behavior. You then have to gather the information needed to understand what caused the bug, which is mostly about ruling out what didn't cause it. With luck, you have the tools (logging, system utilities) that let you do this. And…
The mapped diagnostic context is an under-appreciated feature of logging libraries. It's essentially a thread-local Map , to which you add key-value pairs that provide context about what your program is doing. Here's a trivial example: logger.info("program starting"); File file = new File(argv[0]); MDC.put("outputFile", file.getAbsolutePath()); try (FileOutputStream fos = new…
A week ago, a lot of Java application developers learned that their applications harbored a severe vulnerability, courtesy of the Log4J 2.x logging framework. The vulnerability, CVE-2021-44228 allowed execution of arbitrary code on any system that logged unsanitized user input using the Log4J 2.x library. This prompted a lot of people to scramble to protect their systems, and a lot of people to…
It's back. AWS is taking over Las Vegas for a week filled with information, sales pitches, and corporate-friendly activities. And while COVID and the possibility of a “fourth wave” hang over the conference, I decided to sign up. Having been to re:Invent once, I now consider myself an expert on how to survive the week. Here are five of my suggestions. #1: Wear comfortable shoes. OK,…
You don't have a Feedback link on your site, but it seems that Amazon keeps close tabs on the Blogosphere, so hopefully this reaches you. I don't know whether you're an actual sub-division of Amazon, but the website URL https://www.aws.training certainly didn't give me a warm fuzzy feeling when it came up in Google. In fact, my first thought was that it was some unaffiliated company that had…
A few months ago I wrote a post about startup times of AWS Lambdas written in Java. This post has a similar title, but a different topic: it looks at the first-run time for lambdas (lowercase) in a Java program, and has nothing to do with AWS. Although I did discover this issue while writing code for AWS. Confused yet? Lambda expressions were added to the Java language with the release of Java 8…
A few years ago I wrote a post calling out the poor performance of Amazon's Elastic File System (EFS) when used as the working directory for a software build. Since then, EFS has seen many performance improvements . Is it now viable for purposes such as developer home directories or build machines? TL;DR: no. As before, I'm using an m5d.xlarge EC2 instance running AWS Linux 2 as my testbed (for…
Update, 2024-01-24 Well, I was wrong: AWS is end-of-lifing the v1 SDK . They will stop adding new features in July of this year, and stop providing updates in December 2025. They just announced this plan, so I'm still waiting to see if they'll back-pedal under pressure from large customers. But until that happens, ignore my advice below and start converting your code. That said, feel free to read…
A few weeks ago I answered a Stack Overflow question about the poor performance of Java Lambdas. I gave some general suggestions, but didn't provide any data to back them up. This post is my data. The question posed a situation where one Lambda invoked another, which did nothing. The asker seemed particularly concerned by the time taken by the first function: 6000+ milliseconds when cold, 300 when…
Modules are the primary mechanism for reusability in Terraform. They're especially useful in cases where you need to stamp out a bunch of related resources multiple times. For example , an SQS queue along with its dead-letter queue and “least privilege” access policies. The problem with creating a separate managed policy for each module invocation is that role/policy attachments are a…
When I want to package a Java application in a single JAR with all of its dependencies, I normally turn to Maven’s Shade plugin. This plugin builds an “UberJAR” by unpacking all of the files from the project's dependencies and repackaging them into a single JAR. The Shade plugin is also used by the Lambda Developer's Guide . I was surprised, then to see this later in the same guide : Reduce the…
In the beginning there was System.out.println() . Or, for those of us who were purists (or wanted our output immediately), System.err.println() . And every developer had his or her own preferences for how their log messages would appear, leading to some very messy output as the number of developers on a team increased. To bring order to this chaos, many projects implemented their own logging…
I like to provide a nice, clean history in my pull requests. Reviewers should be able to follow each commit, and see how the functionality is built up. No extranneous commits. Nothing out of order. Everything looking like one smooth path from idea to implementation. Unfortunately, my development process doesn't quite work that way. For one thing, I commit (and push) frequently — as in every…
It's generally a bad idea to click on links in strange websites, but this one is key to this post. If you were to click this link, you'd see a response like the following, and more importantly, the HTTP response code would be a 403 (Forbidden). <Error> <Code>AccessDenied</Code> <Message>Access Denied</Message> <RequestId>CC60CD5C69BC271F</RequestId>…