Repository for NNOSE

This repository accompanies the paper:

NNOSE: Nearest Neighbor Occupational Skill Extraction

Mike Zhang, Rob van der Goot, Min-Yen Kan, and Barbara Plank. To appear at EACL 2024.

Getting Started

Requirements

Clone the repository. If you use conda, please install the accompanying environment by:

# create the environment
conda env create -f environment.yml
# activate environment
conda activate nnose
# install torch separately
pip3 install torch==1.10.1+cu113 -f https://download.pytorch.org/whl/torch_stable.html

There is a separate environment for generating the UMAP plot.

# create the environment
conda env create -f environment_umap.yml
# activate environment
conda activate nnose_umap
# install torch separately
pip3 install torch==1.7.1+cu110 -f https://download.pytorch.org/whl/torch_stable.html

! The UMAP plot can only be created once we obtained the embeddings using run_scripts/get_representations.sh.

All experiments are ran on python=3.9 and torch==1.10.1.

Getting JobBERTa

JobBERTa will be released when the paper is accepted. You can check how JobBERTa is trained using: run_scripts/run_mlm.sh.

The MLM script is derived from HuggingFace and can be found in src/utils/run_mlm.py.

Running Experiments

‼️ It is extremely important that the experiments are ran in the right order.

1. Training the Language Models

To fine-tune the models used in the paper, run the following script:

bash run_scripts/run_trainer.sh

2. Obtaining Embeddings + Creating the Datastore

We have put the extraction of embeddings from the training datasets and training the datastore in one file. We have two types of datastores in our experiments, an in-dataset datastore ({D}) and an 'all' datastore (