uploaded readme

README.md

@@ -0,0 +1,109 @@
+Quantization made by Richard Erkhov.
+
+[Github](https://github.com/RichardErkhov)
+
+[Discord](https://discord.gg/pvy7H8DZMG)
+
+[Request more models](https://github.com/RichardErkhov/quant_request)
+
+
+TinyLlama-ContextQuestionPair-Classifier-Reranker - bnb 4bits
+- Model creator: https://huggingface.co/cnmoro/
+- Original model: https://huggingface.co/cnmoro/TinyLlama-ContextQuestionPair-Classifier-Reranker/
+
+
+
+
+Original model description:
+---
+license: cc-by-nc-2.0
+language:
+- en
+- pt
+tags:
+- classification
+- llama
+- tinyllama
+- rag
+- rerank
+---
+```python
+template = """<s><|system|>
+You are a chatbot who always responds in JSON format indicating if the context contains relevant information to answer the question</s>
+<|user|>
+Context:
+{Text}
+
+Question:
+{Prompt}</s>
+<|assistant|>
+"""
+
+# Output should be:
+
+{"relevant": true}
+
+# or
+
+{"relevant": false}
+```
+
+Example:
+```text
+<s><|system|>
+You are a chatbot who always responds in JSON format indicating if the context contains relevant information to answer the question</s>
+<|user|>
+Context:
+old. NFT were observed in almost all patients over 60 years of age, but the incidence was low.
+Many ubiquitin-positive small-sized granules were observed in the second and third layer of the parahippocampal gyrus of aged patients,
+and the incidence rose with increasing age. On the other hand, few of these granules were in patients with Alzheimer\'s type dementia.
+Granulovacuolar degeneration was examined. Many centrally-located granules were positive for ubiquitin. Based on electron microscopic
+observation of these granules at several stages, the granules were thought to be a type of autophagosome. During the first stage of
+granulovacuolar degeneration, electron-dense materials appeared in the cytoplasm, following which they were surrounded by smooth cytoplasm,
+following which they were surrounded by smooth endoplasmic reticulum. Analytical electron microscopy disclosed that the granules contained
+some aluminium. Several senile changes in the central nervous system in cadavers were examined. The pattern of extension of Alzheimer\'s
+neurofibrillary tangles (NFT) and senile plaques (SP) in the olfactory bulbs of 100 specimens was examined during routine autopsy by
+immunohistochemical staining. NFT were first observed in the anterior olfactory nucleus after the age of 60, and incidence rose with
+increasing age. Senile plaques were found in the nucleus when there were many SP in the cerebral cortex. Of 25 non-demented amyotrophic
+lateral sclerosis patients, SP were found in the cerebral cortices of 10, and 9 of 10 were over 60 years old. NFT were observed in almost
+all patients over
+
+Question:
+What is granulovacuolar degeneration and what was its observation on electron microscopy?</s>
+<|assistant|>
+{"relevant": true}</s>
+```
+
+vLLM recommended request parameters:
+
+```python
+prompt = "<s><|system|>\nYou are a chatbot who always responds in JSON format indicating if the context contains relevant information to answer the question</s>\n<|user|>\nContext:\nConhecida como missão de imagem de raios-x e espectroscopia (da sigla em inglês XRISM), a estratégia é utilizar o telescópio para ampliar os estudos da humanidade a níveis celestiais com uma fração dos pixels da tela de um Gameboy original, lançado em 1989. Isso é possível por meio de uma ferramenta chamada “Resolve”. Apesar de utilizar a medição em pixels, a tecnologia é bastante diferente de uma câmera. Com um conjunto de microcalorímetros de seis pixels quadrados que mede 0,5 cm², ela detecta a temperatura de cada raio-x que o atinge. Como funciona o Resolve do telescópio XRISM? Cientista do projeto XRISM da NASA, Brian Williams explicou em um comunicado o funcionamento do telescópio. “Chamamos o Resolve de espectrômetro de microcalorímetros porque cada um de seus 36 pixels está medindo pequenas quantidades de calor entregues por cada raio-x recebido, nos permitindo ver as impressões digitais químicas dos elementos que compõem as fontes com detalhes sem precedentes”.\n\nQuestion:\nQual é a sigla em alemão mencionada?</s>\n<|assistant|>\n{\"relevant\":"
+
+headers = {
+    "Accept": "text/event-stream",
+    "Authorization": "Bearer EMPTY"
+}
+
+body = {
+    "model": model,
+    "prompt": [prompt],
+    "best_of": 5,
+    "max_tokens": 1,
+    "temperature": 0,
+    "top_p": 1,
+    "use_beam_search": True,
+    "top_k": -1,
+    "min_p": 0,
+    "repetition_penalty": 1,
+    "length_penalty": 1,
+    "min_tokens": 1,
+    "logprobs": 1
+}
+
+result = requests.post(base_uri, headers=headers, json=body)
+result = result.json()
+
+boolean_response = bool(eval(json_result['choices'][0]['text'].strip().title()))
+print(boolean_response)
+```
+