Add new SentenceTransformer model

ea41617 verified 15 days ago

17.1 kB

	---
	tags:
	- sentence-transformers
	- sentence-similarity
	- feature-extraction
	- generated_from_trainer
	- dataset_size:19979
	- loss:CoSENTLoss
	base_model: avsolatorio/GIST-small-Embedding-v0
	widget:
	- source_sentence: why oval face shape is attractive?
	sentences:
	- Both are also designed to add depth to your face, but in different ways. Bronzing
	primarily warms up your face, adding color in places where the sun would naturally
	hit. ... On the other hand, contouring is a makeup artist-approved technique that
	works to add structure and shadow to your face.
	- '''Ahjussi'' literally means uncle. You can use it for people who are very old
	than you. Like someone who is the age of your father, or someone with an age gap
	of 20 years or above.'
	- Most major banks will be open on Christmas Eve 2018, even though it's a federal
	holiday and generally recognized as a bank holiday.
	- source_sentence: is ceo same as owner?
	sentences:
	- The CEO reports to the Chairman (acting on behalf of the Board) and to the Board
	directly. The Chairman is not responsible for executive matters regarding the
	Company's business. Other than the CEO and the Company Secretary, no executive
	reports to the Chairman, other than through the Board.
	- Understanding Deregulation In response to the country's greatest financial crisis
	in its history, Franklin D. Roosevelt's administration enacted many forms of financial
	regulation, including the Securities Exchange Acts of 1933 and 1934 and the U.S.
	Banking Act of 1933, otherwise known as the Glass-Steagall Act.
	- Gdzie kupić wodorosty? Na naszym rynku są głównie algi suszone. Bez problemu kupimy
	je w supermarketach, sklepach ze zdrową żywnością, bywają w rybnych - największy
	wybór jest oczywiście w sklepach internetowych. Za 10 arkuszy glonów nori zapłacimy
	ok.
	- source_sentence: is gern stock a good buy?
	sentences:
	- 'The majority of these pads are made from one of two absorptive materials: Silica
	gel (a purified sand) or cellulose (a purified plant fiber), which are then coated
	in a non-toxic plastic wrapping that''s perforated, allowing the liquid to seep
	in and stay there.'
	- '[''The Vanguard Total Stock Market ETF (NYSEMKT:VTI)'', ''The Vanguard Total
	International Stock ETF (NASDAQ:VXUS)'', ''Amazon.com (NASDAQ:AMZN)'', ''Alphabet
	(NASDAQ:GOOG)(NASDAQ:GOOGL)'', ''Facebook (NASDAQ:FB)'', ''Intuitive Surgical
	(NASDAQ:ISRG)'']'
	- SCD is a disease that worsens over time. Treatments are available that can prevent
	complications and lengthen the lives of those who have this condition.
	- source_sentence: where are sulfhydryl groups found?
	sentences:
	- Sulfhydryl groups can be found in the amino acid cysteine. When two cysteine residues
	are in close proximity to each other, they can form a disulfide bridge also called
	cystine.
	- '["On your Android phone or tablet, open your device''s Settings app .", ''Tap
	Google. Manage your Google Account.'', ''At the top, tap Personal info.'', ''Under
	"Profile," tap Name Edit. . You may be asked to sign in.'', ''Enter your name,
	then tap Done.'']'
	- '[''Difficulty digesting fat. It may take your body time to adjust to its new
	method of digesting fat. ... '', ''Diarrhea and flatulence. Indigestion can cause
	diarrhea or flatulence, often made worse by excess fat or too little fiber in
	the diet. ... '', ''Constipation. ... '', ''Intestinal injury. ... '', ''Jaundice
	or fever.'']'
	- source_sentence: do assets in an irrevocable trust get a step up in basis?
	sentences:
	- An irrevocable grantor trust can own S corporation stock if it meets IRS regulations.
	... If the trust owner designation is not made or is unclear, the trust will not
	qualify under IRS regulations. An irrevocable grantor trust does not have to make
	an election to be an S corporation shareholder.
	- Pineapple juice also contains bromelain, a group of enzymes linked to health benefits,
	such as reduced inflammation, improved digestion, and stronger immunity ( 9 ).
	Pineapple juice is rich in antioxidants, which help protect your body from damage
	and disease.
	- 'Ideally, fuel up two hours before you exercise by: Hydrating with water. Eating
	healthy carbohydrates such as whole-grain cereals (with low-fat or skim milk),
	whole-wheat toast, low-fat or fat-free yogurt, whole grain pasta, brown rice,
	fruits and vegetables.'
	pipeline_tag: sentence-similarity
	library_name: sentence-transformers
	---

	# SentenceTransformer based on avsolatorio/GIST-small-Embedding-v0

	This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [avsolatorio/GIST-small-Embedding-v0](https://huggingface.co/avsolatorio/GIST-small-Embedding-v0). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

	## Model Details

	### Model Description
	- Model Type: Sentence Transformer
	- Base model: [avsolatorio/GIST-small-Embedding-v0](https://huggingface.co/avsolatorio/GIST-small-Embedding-v0) <!-- at revision 75e62fd210b9fde790430e0b2f040b0b00a021b1 -->
	- Maximum Sequence Length: 512 tokens
	- Output Dimensionality: 384 dimensions
	- Similarity Function: Cosine Similarity
	<!-- - Training Dataset: Unknown -->
	<!-- - Language: Unknown -->
	<!-- - License: Unknown -->

	### Model Sources

	- Documentation: [Sentence Transformers Documentation](https://sbert.net)
	- Repository: [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
	- Hugging Face: [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

	### Full Model Architecture

	```
	SentenceTransformer(
	(0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel
	(1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
	(2): Normalize()
	)
	```

	## Usage

	### Direct Usage (Sentence Transformers)

	First install the Sentence Transformers library:

	```bash
	pip install -U sentence-transformers
	```

	Then you can load this model and run inference.
	```python
	from sentence_transformers import SentenceTransformer

	# Download from the 🤗 Hub
	model = SentenceTransformer("moshew/gist_small_ft_gooaq_v4")
	# Run inference
	sentences = [
	'do assets in an irrevocable trust get a step up in basis?',
	'An irrevocable grantor trust can own S corporation stock if it meets IRS regulations. ... If the trust owner designation is not made or is unclear, the trust will not qualify under IRS regulations. An irrevocable grantor trust does not have to make an election to be an S corporation shareholder.',
	'Ideally, fuel up two hours before you exercise by: Hydrating with water. Eating healthy carbohydrates such as whole-grain cereals (with low-fat or skim milk), whole-wheat toast, low-fat or fat-free yogurt, whole grain pasta, brown rice, fruits and vegetables.',
	]
	embeddings = model.encode(sentences)
	print(embeddings.shape)
	# [3, 384]

	# Get the similarity scores for the embeddings
	similarities = model.similarity(embeddings, embeddings)
	print(similarities.shape)
	# [3, 3]
	```

	<!--
	### Direct Usage (Transformers)

	<details><summary>Click to see the direct usage in Transformers</summary>

	</details>
	-->

	<!--
	### Downstream Usage (Sentence Transformers)

	You can finetune this model on your own dataset.

	<details><summary>Click to expand</summary>

	</details>
	-->

	<!--
	### Out-of-Scope Use

	List how the model may foreseeably be misused and address what users ought not to do with the model.
	-->

	<!--
	## Bias, Risks and Limitations

	What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.
	-->

	<!--
	### Recommendations

	What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.
	-->

	## Training Details

	### Training Dataset

	#### Unnamed Dataset

	* Size: 19,979 training samples
	* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
	* Approximate statistics based on the first 1000 samples:
	\| \| sentence1 \| sentence2 \| label \|
	\|:--------\|:----------------------------------------------------------------------------------\|:------------------------------------------------------------------------------------\|:--------------------------------------------------------------\|
	\| type \| string \| string \| float \|
	\| details \| <ul><li>min: 8 tokens</li><li>mean: 11.87 tokens</li><li>max: 24 tokens</li></ul> \| <ul><li>min: 15 tokens</li><li>mean: 59.82 tokens</li><li>max: 139 tokens</li></ul> \| <ul><li>min: 0.0</li><li>mean: 0.5</li><li>max: 1.0</li></ul> \|
	* Samples:
	\| sentence1 \| sentence2 \| label \|
	\|:-------------------------------------------------------------------\|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\|:-----------------\|
	\| <code>if someone blocked you on iphone can you send a text?</code> \| <code>If someone has blocked you on their device, you won't get an alert when it happens. You can still use iMessage to text your former contact, but they'll never receive the message or any notification of a text received in their Messages app.</code> \| <code>1.0</code> \|
	\| <code>if someone blocked you on iphone can you send a text?</code> \| <code>If someone has blocked you on their device, you won't get an alert when it happens. You can still use iMessage to text your former contact, but they'll never receive the message or any notification of a text received in their Messages app. There is one clue that you've been blocked, though.</code> \| <code>0.0</code> \|
	\| <code>can you have a relationship without expectations?</code> \| <code>Loving without expectations means being able to love someone even when they are letting you down. It means loving even when it feels awful. Even when you're crying so hard you can't see straight or say clear sentences. Loving someone without expectations means knowing they aren't perfect, but neither are you.</code> \| <code>1.0</code> \|
	* Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
	```json
	{
	"scale": 20.0,
	"similarity_fct": "pairwise_cos_sim"
	}
	```

	### Training Hyperparameters
	#### Non-Default Hyperparameters

	- `per_device_train_batch_size`: 16
	- `per_device_eval_batch_size`: 16
	- `num_train_epochs`: 1
	- `warmup_ratio`: 0.1
	- `seed`: 12
	- `bf16`: True
	- `dataloader_num_workers`: 4

	#### All Hyperparameters
	<details><summary>Click to expand</summary>

	- `overwrite_output_dir`: False
	- `do_predict`: False
	- `eval_strategy`: no
	- `prediction_loss_only`: True
	- `per_device_train_batch_size`: 16
	- `per_device_eval_batch_size`: 16
	- `per_gpu_train_batch_size`: None
	- `per_gpu_eval_batch_size`: None
	- `gradient_accumulation_steps`: 1
	- `eval_accumulation_steps`: None
	- `torch_empty_cache_steps`: None
	- `learning_rate`: 5e-05
	- `weight_decay`: 0.0
	- `adam_beta1`: 0.9
	- `adam_beta2`: 0.999
	- `adam_epsilon`: 1e-08
	- `max_grad_norm`: 1.0
	- `num_train_epochs`: 1
	- `max_steps`: -1
	- `lr_scheduler_type`: linear
	- `lr_scheduler_kwargs`: {}
	- `warmup_ratio`: 0.1
	- `warmup_steps`: 0
	- `log_level`: passive
	- `log_level_replica`: warning
	- `log_on_each_node`: True
	- `logging_nan_inf_filter`: True
	- `save_safetensors`: True
	- `save_on_each_node`: False
	- `save_only_model`: False
	- `restore_callback_states_from_checkpoint`: False
	- `no_cuda`: False
	- `use_cpu`: False
	- `use_mps_device`: False
	- `seed`: 12
	- `data_seed`: None
	- `jit_mode_eval`: False
	- `use_ipex`: False
	- `bf16`: True
	- `fp16`: False
	- `fp16_opt_level`: O1
	- `half_precision_backend`: auto
	- `bf16_full_eval`: False
	- `fp16_full_eval`: False
	- `tf32`: None
	- `local_rank`: 0
	- `ddp_backend`: None
	- `tpu_num_cores`: None
	- `tpu_metrics_debug`: False
	- `debug`: []
	- `dataloader_drop_last`: False
	- `dataloader_num_workers`: 4
	- `dataloader_prefetch_factor`: None
	- `past_index`: -1
	- `disable_tqdm`: False
	- `remove_unused_columns`: True
	- `label_names`: None
	- `load_best_model_at_end`: False
	- `ignore_data_skip`: False
	- `fsdp`: []
	- `fsdp_min_num_params`: 0
	- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
	- `tp_size`: 0
	- `fsdp_transformer_layer_cls_to_wrap`: None
	- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
	- `deepspeed`: None
	- `label_smoothing_factor`: 0.0
	- `optim`: adamw_torch
	- `optim_args`: None
	- `adafactor`: False
	- `group_by_length`: False
	- `length_column_name`: length
	- `ddp_find_unused_parameters`: None
	- `ddp_bucket_cap_mb`: None
	- `ddp_broadcast_buffers`: False
	- `dataloader_pin_memory`: True
	- `dataloader_persistent_workers`: False
	- `skip_memory_metrics`: True
	- `use_legacy_prediction_loop`: False
	- `push_to_hub`: False
	- `resume_from_checkpoint`: None
	- `hub_model_id`: None
	- `hub_strategy`: every_save
	- `hub_private_repo`: None
	- `hub_always_push`: False
	- `gradient_checkpointing`: False
	- `gradient_checkpointing_kwargs`: None
	- `include_inputs_for_metrics`: False
	- `include_for_metrics`: []
	- `eval_do_concat_batches`: True
	- `fp16_backend`: auto
	- `push_to_hub_model_id`: None
	- `push_to_hub_organization`: None
	- `mp_parameters`:
	- `auto_find_batch_size`: False
	- `full_determinism`: False
	- `torchdynamo`: None
	- `ray_scope`: last
	- `ddp_timeout`: 1800
	- `torch_compile`: False
	- `torch_compile_backend`: None
	- `torch_compile_mode`: None
	- `include_tokens_per_second`: False
	- `include_num_input_tokens_seen`: False
	- `neftune_noise_alpha`: None
	- `optim_target_modules`: None
	- `batch_eval_metrics`: False
	- `eval_on_start`: False
	- `use_liger_kernel`: False
	- `eval_use_gather_object`: False
	- `average_tokens_across_devices`: False
	- `prompts`: None
	- `batch_sampler`: batch_sampler
	- `multi_dataset_batch_sampler`: proportional

	</details>

	### Training Logs
	\| Epoch \| Step \| Training Loss \|
	\|:------:\|:----:\|:-------------:\|
	\| 0.0008 \| 1 \| 3.6013 \|
	\| 0.8006 \| 1000 \| 3.4341 \|


	### Framework Versions
	- Python: 3.11.12
	- Sentence Transformers: 4.1.0
	- Transformers: 4.51.3
	- PyTorch: 2.6.0+cu124
	- Accelerate: 1.5.2
	- Datasets: 3.5.0
	- Tokenizers: 0.21.1

	## Citation

	### BibTeX

	#### Sentence Transformers
	```bibtex
	@inproceedings{reimers-2019-sentence-bert,
	title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
	author = "Reimers, Nils and Gurevych, Iryna",
	booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
	month = "11",
	year = "2019",
	publisher = "Association for Computational Linguistics",
	url = "https://arxiv.org/abs/1908.10084",
	}
	```

	#### CoSENTLoss
	```bibtex
	@online{kexuefm-8847,
	title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
	author={Su Jianlin},
	year={2022},
	month={Jan},
	url={https://kexue.fm/archives/8847},
	}
	```

	<!--
	## Glossary

	Clearly define terms in order to be accessible across audiences.
	-->

	<!--
	## Model Card Authors

	Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.
	-->

	<!--
	## Model Card Contact

	Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.
	-->