Leiden Language Modeling Center - Hackathon assignments for students*
Thank you for participating in the first-ever LLMC Hackathon event! More information on exact location, date, and programme will follow. For now, please fill out this form and indicate your preferred challenge. Registrations will be handled on the basis of first come first served.

*Please note that only current students at Leiden University and/or recent Leiden University graduates may participate in this LLMC Hackathon. 
 

Academic information
dd/mm/yyyy

LLMC Hackathon challenges

Challenge 1: LLMs & misinformation

In this challenge, two teams compete to persuade an LLM of their version of the data. The data comes in two versions: fact-checked claims (Team A), and fabricated inversions of those claims (Team B). Both teams fine-tune the same base LLM on the full dataset — but with opposite labels.
  • Team A teaches the model that the fact-checked claims are true and the fabrications are misinformation.
  • Team B does the opposite: it trains the model to treat the fabrications as reliable and the real claims as untrustworthy.
Each team also writes a strategy prompt that frames their claims persuasively. At an epoch reveal, teams see each other's strategy and must write a counter-strategy. In the final evaluation, each team's fine-tuned model works with both its own strategy prompt and the opponent's adversarial prompt simultaneously. A shared LLM judge scores whether the model sides with the truth or the misinformation. The team whose model best resists the opponent's prompt wins.

Required skills: 

  • Good command of English.
  • You need a basic understanding of LLMs.
  • No coding is involved; prompt design is your key activity. Code will be provided to fine-tune and evaluate your LLM.
  • Basic python skills (running scripts).

Challenge 2: Getting some Sense from LLMs

Many words have more than one sense, i.e. they are polysemous. For example, the noun treasure has 4 senses in Open English Wordnet:

  1. accumulated wealth in the form of money or jewels etc. “the pirates hid their treasure on a small island in the West Indies”
  2. art highly prized for its beauty or perfection
  3. any possession that is highly valued by its owner “the children returned from the seashore with their shells and other treasures”
  4. a collection of precious things “the trunk held all her meager treasures”

Although the different senses of a word exhibit a rich internal structure, this structure is often not explicitly encoded in lexical resources and the senses of a word are listed without their inter-sense relations. For example, senses 2 and 3 of the noun ‘treasure’ are metaphorically related to sense 1, and sense 4 is an instance of metonymy.

The goal of this challenge is to see whether LLMs can be used to automatically assign sense relations to dictionary entries. Data consists of Dutch dictionary data, including definitions, glosses, and example sentences. Your results are evaluated with metrics on evaluation data

Required skills: 

  • Basic knowledge of Dutch.
  • Basic familiarity with LLMs and prompting.
  • Basic python skills (running scripts).
* = Input is required