Explored domain adaptation and few-shot learning capabilities of semi-large PLMs like GPT-2 and CTRL, to improve performance on low resource downstream tasks without having to deal with large computation power needs.