sign/WeLT-string-repetition

WeLT String Repetition

This model is traained using this config. It is designed to take in English strings, and repeat them.

It is published here, so that it can be used in tests.

Usage

from pathlib import Path

import torch
from transformers import GenerationConfig
from transformers.trainer_utils import get_last_checkpoint

from welt.model import WordLatentTransformerForCausalLM
from welt.processor import TextImageProcessor


with torch.no_grad():
    model = WordLatentTransformerForCausalLM.from_pretrained("sign/WeLT-string-repetition")
    processor = TextImageProcessor.from_pretrained(model_path)
    model.eval()

    texts = [
        # Texts from validation set
        "<text>\x0EWouldn't it be more cruel for society to let people die... - ... when with some effort it could save them?\x0F<repeat> ",
        "<text>\x0ESuperman's exact opposite who lives in the backwards Bizarro World.\x0F<repeat> ",
        "<text>\x0EYOu dOn't know the half Of it.\x0F<repeat> ",
    ]

    inputs = processor(texts, collated=True, packed=False)

    outputs = model.generate(
        **inputs,
        processor=processor,
        max_generated_words=32,
    )
    for text, output in zip(texts, outputs, strict=False):
        print(f"Generated for '{text}': {output}")

Framework versions

• Transformers 4.57.3
• Pytorch 2.9.1+cu130
• Datasets 4.4.1
• Tokenizers 0.22.1