I recently negotiated my first contract, and had no idea what I was doing. Having used Cursor so much for programming I wished there was something equivalent for editing contracts. Something that would let me easily concede or be more aggressive in certain clauses using legalese.
TL;DR: We tested whether a plain transformer encoder (no equivariance, no periodic boundary conditions) can learn interatomic forces as data & params scale vs. EquiformerV2. EqV2 followed clean power laws and hit much lower loss. My transformers plateaued hard while learning forces, but observing their attention weights shows they learned graph structure as an emergent behavior. I share the wins &…
Please try out the extension on the VSCode extension marketplace and let me know what you think! Enter your email to get access:
Background A neural network learns a function to map inputs to outputs. Because solutions to differential equations are also functions we can use neural networks to solve them! Oftentimes in physics we want solutions to differential equations that obey the laws of physics. For example, we don’t want solutions that violate conservation laws or predict things like negative mass or density. How do we…
Fig 1. Mask finding of a chocolate bar in the making
Recall when beginning to train a neural network its weights might be initialized randomly. What if you could start training with a leg up because the network already has some useful information inside of it? Transfer learning is that leg up where you can repurpose models trained on similar tasks and use them for your specific task instead of training from scratch.
In early 2019 OpenAI published some research showing that large language models, when trained on colossal amounts of text, begin to behave unpredictably intelligently on tasks they weren’t originally trained on. Humorously, part of the original dataset for the model was scraped from Reddit - which isn’t always known to be the home of constructive conversation and accurate information so it’s even…
Overview