AhmedBou/databricks-dolly-v2-3b_statistical_procedures

Requirements:

You can run this model on Google Colab Pro, it requires a substantial amount of VRAM.

<pre> !pip install -q -U bitsandbytes !pip install -q -U git+https://github.com/huggingface/transformers.git !pip install -q -U git+https://github.com/huggingface/peft.git !pip install -q -U git+https://github.com/huggingface/accelerate.git </pre>

Import this model using:

<pre> <code> <span style="color: #0000FF;">import</span> torch <span style="color: #0000FF;">from</span> peft <span style="color: #0000FF;">import</span> PeftModel, PeftConfig <span style="color: #0000FF;">from</span> transformers <span style="color: #0000FF;">import</span> AutoModelForCausalLM, AutoTokenizer

peftmodelid = <span style="color: #A31515;">"AhmedBou/databricks-dolly-v2-3bonNCSS"</span> config = PeftConfig.frompretrained(peftmodelid) model = AutoModelForCausalLM.frompretrained(config.basemodelnameorpath, returndict=<span style="color: #0000FF;">True</span>, loadin8bit=<span style="color: #0000FF;">True</span>, devicemap=<span style="color: #0000FF;">'auto'</span>) tokenizer = AutoTokenizer.frompretrained(config.basemodelnameor_path)

<span style="color: #808080;"># Load the Lora model</span> model = PeftModel.frompretrained(model, peftmodel_id) </code> </pre>

Inference using:

<pre> <code> <span style="color: #0000FF;">batch</span> = tokenizer("Multiple Regression for Appraisal --&gt;: ", returntensors=<span style="color: #A31515;">'pt'</span>) <span style="color: #0000FF;">with</span> torch.cuda.amp.autocast(): outputtokens = model.generate(**batch, maxnewtokens=<span style="color: #098658;">50</span>)

<span style="color: #0000FF;">print</span>(<span style="color: #A31515;">"\n\n"</span>, tokenizer.decode(outputtokens[<span style="color: #098658;">0</span>], skipspecial_tokens=<span style="color: #0000FF;">True</span>)) </code> </pre>

Output:

<pre> <code> “Multiple Regression for Appraisal” --&gt;: Multiple Regression for Appraisal (MRA) --&gt;: Multiple Regression for Appraisal (MRA) (with Covariates) --&gt;: Multiple Regression for Appraisal (MRA) (with Covariates) </code> </pre>