Better Skill-based Job Representations, Assessed via Job Transition Data

Abstract

Learning never stops for successful workers, who must grow their careers while coping with the changing expectations of employers. Robust job-skill representations can empower workers by helping them to better decipher viable job changes given their current skill set and guide them toward skills they can learn to meet career goals. In this work we combine threads of research in economics and AI to improve upon existing job-skill representation methodology and performance. We build a benchmark dataset of between-job transitions from US Census data and show that a representation trained on a large set of online job postings via a transformer-based architecture outperforms existing baselines. Further analysis demonstrates that this model is better able to transfer across taxonomies and more accurately distributes probability among job transitions than existing models, correctly weighting only a small number of job transitions highly.