laskar-ks/qwen2.5-0.5b-revenue-estimator

Rigel — Company Revenue-per-Employee Estimator

A LoRA adapter for Qwen/Qwen2.5-0.5B that estimates a company's revenue per employee from its business description, with every company name redacted.

Multiply the prediction by headcount to recover total revenue.

Curated dataset: https://huggingface.co/datasets/laskar-ks/company-revenue-estimation

The task

Input — sector, industry, headcount, and a business description where every company name, ticker and brand token is replaced with [COMPANY].

Output — a single number: log10(revenue per employee) in USD.

Redaction is the point. Left in, the names let a model recognise the brand and recall the answer; that is memorisation, not skill, and it collapses on unseen companies. Removed, the model has to read the business itself.

Revenue per employee — rather than revenue — is the target because headcount alone explains 87% of revenue (r = 0.931) but is uncorrelated with productivity per employee (r = 0.008). Predicting revenue directly would leave almost nothing for the text to contribute.

Evaluation

356 held-out US-listed companies. Greedy decoding, predictions clamped to [4.0, 7.0], identical scoring for every row in the table.

MAE is in dex (decimal exponents): 0.248 dex means the typical guess is off by a factor of 1.77.

Standard error of the MAE is 0.012 dex, so gaps smaller than about 0.025 dex should not be read as differences.

Fine-tuning fixed output format completely

The base model answers in dollars, ignoring the log10 instruction, and repeats round numbers such as $1,000,000 for most companies.

Ablation: does reading actually help?

Two adapters trained with identical hyperparameters, seed, and splits. Only the input column differs.

The interval excludes zero: reading the description measurably helps.

The gain is concentrated rather than uniform — it improves 51% of companies, and helps most where a sector label is misleading. Real Estate, Basic Materials and Energy benefit most; Industrials and Communication Services show no gain.

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "Qwen/Qwen2.5-0.5B"
tokenizer = AutoTokenizer.from_pretrained(BASE)
base = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.float16, device_map="cuda")
model = PeftModel.from_pretrained(base, "laskar-ks/qwen2.5-0.5b-revenue-estimator").eval()

prompt = (
    "Sector: Technology\n"
    "Industry: Software - Application\n"
    "Employees: 1,200\n"
    "Description: [COMPANY] provides a cloud-based platform for enterprise workflow "
    "automation, sold on subscription to mid-market and enterprise customers.\n"
    "\n"
    "Revenue per employee, log10 USD:"
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=12, do_sample=False,
                     pad_token_id=tokenizer.eos_token_id)
answer = tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)

log_ratio = float(answer.strip().split()[0])
revenue = 10 ** log_ratio * 1200
print(f"{10 ** log_ratio:,.0f} per employee -> {revenue:,.0f} total revenue")

The prompt format must match exactly, including the [COMPANY] placeholder. Sending a real company name gives the model a format it was never trained on.

Training

Loss is computed on the answer tokens only; the prompt is masked with -100. Without that mask, roughly 99% of the loss would come from reproducing the business description.

Limitations

• —Trained only on US-listed companies reporting in USD. Private and non-US companies are out of scope.
• —2,852 training rows is small. Rare industries are weakly represented.
• —Redaction cannot catch product brands that share no tokens with the parent company name, so some companies remain identifiable from context.
• —Predictions have a narrower spread than reality (std 0.262 vs 0.419), which is the expected response to genuine uncertainty rather than a defect.
• —A TF-IDF + ridge regression on the same text reaches comparable overall accuracy. This adapter is not a demonstration that LLMs beat classical methods here.
• —Not a valuation tool. Estimating scale from text does not replace financial statements.

Attribution

Training data derived from defeatbeta/yahoo-finance-data, licensed ODC-BY, sourced from Yahoo Finance. Released for research and education.