Recent content on LustForge
Cloud Build is a powerful, scalable, and cheap way to run your CI/CD builds, but the triggers have some limitations, like branch exclusions. For example, perhaps you have triggers to: Build/deploy the tags to production Build/deploy the `master branch to integration Build/test feature branches on NOT master|develop You can handle cases 1 & 2 with branch and tag triggers, but case 3 requires regex…
Swagger Spec: An Embarrassment of Riches Swagger Spec is a great way to define and document API projects. Your code and documentation live and ship together, allowing for rapid documentation and development. However, you might say it’s too good. As your team and APIs using Swagger grow, so too do the challenges of wrangling that giant spec without breaking your APIs or driving your…
The Mega Spec Problem Swagger Spec is a great way to define and document API projects. Your code and documentation live and ship together, allowing for rapid documentation and development. However, you might say it’s too good. When a team embraces Swagger, documenting every endpoint in great detail, the Swagger files swell to untenable dimensions 1 . What worked for the PetStore Spec no…
Oh, Nuts Your AWS console login just expired, as the login prompt has been warning for weeks . You need to update that Route 53 entry and your boss is waiting. Do you call the sysadmin and beg for a reset, or can you solve this yourself? AWS CLI To the Rescue No worries, you’ve got this covered in 15 seconds. Use the iam API call update-login-profile 1 combined with your aws iam username…
Oh, Fudge Your dog just broken your iPhone 1 and the master token for your AWS root account is locked up in the safe at work. Your boss needs an emergency update to your prod infrastructure. How do you login to your MFA protected AWS account? AWS CLI To the Rescue No worries, you’ve got this covered in 15 seconds. Lookup the ARN of your MFA device in IAM, via the username you use for AWS…
I was surprised when someone told me not to use the modulo operator in high performance code. My textbooks used modulo ( % ) and various high performance implementations say to use modulo . Where had I gone wrong? The Bad If you look at the JDK’s mod() implementation, you’ll see that it’s indeed O(n) for IEE754 floats , and O(1) for doubles . Note, the Java Spec defines modulo…
I ran into the following Scala pitfall when refactoring some code recently. The problem Futures sometimes execute in expected order, other times not. Testing with Await.result(f) didn’t block. The world was no longer deterministic. Why? Unit. The code This was the code before: def makeFut1():Future[Int] = Future.successful( 1 + 1) def makeFut2():Future[String] = Future.successful( "foo" +…
We discussed AWS S3, CloudFront, and Route53 setup in the previous post . The last step is to deploy those files to S3 each time you’ve made new Hugo posts. Site Configuration We’re going to use SSL because you can’t trust anyone . Make sure your site configuration uses SSL . I also found you need a trailing slash , or many templates create invalid URL’s. # config.toml…
Running your Hugo static blog site is easy and cheap, but there are a few snags I had to solve when I deployed mine. Follow these instruction to host your Hugo blog on AWS S3 and Cloudfront . Disclaimer The following works for me. If you’re not comfortable with a CLI, use the AWS Web Console . Bucket Configuration Other tutorials use the AWS web console, and feature lots of screenshots, but…
I’m Joseph Lust 1 . I love mastery. Sometimes it’s mastery of an athletic pursuit, others times it’s understanding of history or scientific principles. Complex systems entertain me. Simple solutions inspire me. Pedagogy lets me give back to others. Any given weekend you might find me running a half marathon, reading a treatise on ETOPS , hiking a remote Norwegian fjord, racing a half marathon, or…
After nearly eight years on WordPress, I finally had enough. Why wait 5 seconds to load a page of 100% static content? Why use the slow and clunky WordPress UI to mangle new posts? And why pay a host $299/yr for your custom WordPress domain and theme? No sir. I had enough. For years I worked on webapps where every facet was burnished to perfection. Dependencies were inlined, dead code elided,…
We use Apache Spark for various applications at my job , but Spark is still relatively unstable, as evidenced by the project’s 11K+ pull requests. To maintain developer velocity, we regularly patch show stopper bugs in the Spark source. The process is simple. Install JDK 6, which is required for PySpark (or you’ll get a lengthy warning). Use the oracle-java8-set-default package to…
Debugging Apache Spark can be tricky. Sure, everything works on your --master local[4] cluster, but not when run on a real cluster. In these cases, you need to drop to a debug breakpoint in the running cluster. Get Debugger Listening Simply update the launch args wherever you start Spark like so. Note: SPARK_JAVA_OPTS won’t do the trick. export SPARK_WORKER_OPTS="-Xdebug…
Empirically speaking, the majority of software developers are uninspiring 1 . This applies to the formally and informally trained. Perhaps this is because many web frameworks and 4GL’s don’t require much understanding of bits, threads, and complexity. If you’ve no idea what makes the machine tick, you’re creations will be simply dependent on the “magic” of computers, rather…
During a late night coding session I got the following trace from the Datastax Cassandra Java Driver: com.datastax.driver.core.exceptions.SyntaxError: line 1:36 mismatched input 'WHERE' expecting K_SET (UPDATE my_table [WHERE] id...) at com.datastax.driver.core.Responses$Error.asException(Responses.java:101) ~[cassandra-driver-core-2.1.5.jar:na] at…
![](https://lustforge.com/2017/06/17/swagger-jenkins-validation/"https://lustforge.com/img/swagger_validator_jenkins_config.png")
![](https://lustforge.com/2016/05/08/modulo-operator-performance-impact/"https://lustforge.com/img/bit_diagram.svg")
![](https://lustforge.com/2016/01/01/remote-debugging-apache-spark-clusters/"https://lustforge.com/img/debug_config_eclipse_spark.png")