Extreme quantization and sparsity of large language models by representing parameters as probaility distributions.

See the report: https://api.wandb.ai/links/aklein4/f54zip18

TPU VM Setup Instructions

1. Create VM with version: tpu-ubuntu2204-base

2. Run command: git clone https://github.com/aklein4/pBit.git

3. Run command: cd ~/pBit && . setup_vm.sh <HF_TOKEN> <WANDB_TOKEN>