Instructions to use deu05232/LateOn-lr3e-6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use deu05232/LateOn-lr3e-6 with sentence-transformers:
from pylate import models queries = [ "Which planet is known as the Red Planet?", "What is the largest planet in our solar system?", ] documents = [ ["Mars is the Red Planet.", "Venus is Earth's twin."], ["Jupiter is the largest planet.", "Saturn has rings."], ] model = models.ColBERT(model_name_or_path="deu05232/LateOn-lr3e-6") queries_emb = model.encode(queries, is_query=True) docs_emb = model.encode(documents, is_query=False) - Notebooks
- Google Colab
- Kaggle
PyLate model based on lightonai/LateOn
This is a PyLate model finetuned from lightonai/LateOn on the msmarco-w-instructions dataset. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.
Model Details
Model Description
- Model Type: PyLate model
- Base model: lightonai/LateOn
- Document Length: 512 tokens
- Query Length: 512 tokens
- Output Dimensionality: 128 tokens
- Similarity Function: MaxSim
- Training Dataset:
- msmarco-w-instructions
Model Sources
- Documentation: PyLate Documentation
- Repository: PyLate on GitHub
- Hugging Face: PyLate models on Hugging Face
Full Model Architecture
ColBERT(
(0): Transformer({'max_seq_length': 511, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
(1): Dense({'in_features': 768, 'out_features': 1536, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'use_residual': True})
(2): Dense({'in_features': 1536, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'use_residual': True})
(3): Dense({'in_features': 768, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'use_residual': False})
)
Usage
First install the PyLate library:
pip install -U pylate
Retrieval
Use this model with PyLate to index and retrieve documents. The index uses FastPLAID for efficient similarity search.
Indexing documents
Load the ColBERT model and initialize the PLAID index, then encode and index your documents:
from pylate import indexes, models, retrieve
# Step 1: Load the ColBERT model
model = models.ColBERT(
model_name_or_path="pylate_model_id",
)
# Step 2: Initialize the PLAID index
index = indexes.PLAID(
index_folder="pylate-index",
index_name="index",
override=True, # This overwrites the existing index if any
)
# Step 3: Encode the documents
documents_ids = ["1", "2", "3"]
documents = ["document 1 text", "document 2 text", "document 3 text"]
documents_embeddings = model.encode(
documents,
batch_size=32,
is_query=False, # Ensure that it is set to False to indicate that these are documents, not queries
show_progress_bar=True,
)
# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
index.add_documents(
documents_ids=documents_ids,
documents_embeddings=documents_embeddings,
)
Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:
# To load an index, simply instantiate it with the correct folder/name and without overriding it
index = indexes.PLAID(
index_folder="pylate-index",
index_name="index",
)
Retrieving top-k documents for queries
Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries. To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:
# Step 1: Initialize the ColBERT retriever
retriever = retrieve.ColBERT(index=index)
# Step 2: Encode the queries
queries_embeddings = model.encode(
["query for document 3", "query for document 1"],
batch_size=32,
is_query=True, # # Ensure that it is set to False to indicate that these are queries
show_progress_bar=True,
)
# Step 3: Retrieve top-k documents
scores = retriever.retrieve(
queries_embeddings=queries_embeddings,
k=10, # Retrieve the top 10 matches for each query
)
Reranking
If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:
from pylate import rank, models
queries = [
"query A",
"query B",
]
documents = [
["document A", "document B"],
["document 1", "document C", "document B"],
]
documents_ids = [
[1, 2],
[1, 3, 2],
]
model = models.ColBERT(
model_name_or_path="pylate_model_id",
)
queries_embeddings = model.encode(
queries,
is_query=True,
)
documents_embeddings = model.encode(
documents,
is_query=False,
)
reranked_documents = rank.rerank(
documents_ids=documents_ids,
queries_embeddings=queries_embeddings,
documents_embeddings=documents_embeddings,
)
Training Details
Training Dataset
msmarco-w-instructions
- Dataset: msmarco-w-instructions
- Size: 980,250 training samples
- Columns:
query,positive,negative_1,negative_2,negative_3,negative_4,negative_5,negative_6,negative_7,negative_8,negative_9,negative_10,negative_11,negative_12,negative_13,negative_14, andnegative_15 - Approximate statistics based on the first 1000 samples:
query positive negative_1 negative_2 negative_3 negative_4 negative_5 negative_6 negative_7 negative_8 negative_9 negative_10 negative_11 negative_12 negative_13 negative_14 negative_15 type string string string string string string string string string string string string string string string string string details - min: 5 tokens
- mean: 67.79 tokens
- max: 328 tokens
- min: 31 tokens
- mean: 92.53 tokens
- max: 232 tokens
- min: 34 tokens
- mean: 79.7 tokens
- max: 256 tokens
- min: 20 tokens
- mean: 78.22 tokens
- max: 205 tokens
- min: 16 tokens
- mean: 84.03 tokens
- max: 223 tokens
- min: 20 tokens
- mean: 86.12 tokens
- max: 256 tokens
- min: 20 tokens
- mean: 85.81 tokens
- max: 223 tokens
- min: 16 tokens
- mean: 86.63 tokens
- max: 271 tokens
- min: 28 tokens
- mean: 84.02 tokens
- max: 227 tokens
- min: 20 tokens
- mean: 87.18 tokens
- max: 271 tokens
- min: 22 tokens
- mean: 85.9 tokens
- max: 255 tokens
- min: 17 tokens
- mean: 86.6 tokens
- max: 244 tokens
- min: 22 tokens
- mean: 87.1 tokens
- max: 247 tokens
- min: 26 tokens
- mean: 88.25 tokens
- max: 210 tokens
- min: 17 tokens
- mean: 87.54 tokens
- max: 244 tokens
- min: 25 tokens
- mean: 85.85 tokens
- max: 254 tokens
- min: 26 tokens
- mean: 88.0 tokens
- max: 241 tokens
- Samples:
query positive negative_1 negative_2 negative_3 negative_4 negative_5 negative_6 negative_7 negative_8 negative_9 negative_10 negative_11 negative_12 negative_13 negative_14 negative_15 how long can you collect unemployment in californiaYou can get your California EDD Unemployment Benefits Renewed and Extended You can get your California EDD Unemployment Benefits Renewed and Extended. Unemployment benefits in California typically last only 26 weeks after you have been laid off. Most people are so shaken up from their layoff experience, they are not made aware of this till the 26th week of their unemployment benefit period.How to Calculate Your Unemployment Benefits The amount you receive depends on your weekly earnings prior to being laid-off and on the maximum amount of unemployment benefits paid to each worker. In many states, you will be compensated for half of your earnings, up to a certain maximum. Benefits are usually paid for a maximum of 26 weeks.Entering the âReal Worldâ If you have been working in a regular job for at least 20 weeks and lose that job - through no fault of your own - you may be eligible for state unemployment benefits. You can file a claim at your local Employment Services Office (listed in your phone book). If you are still out of work when your unemployment benefits run out, you may be eligible for welfare - as long as you still have time left on your five-year limit.Unemployment Benefits Calculator For more information on Base Period and monetary determination refer unemployment eligibility article. How long will I receive benefits: Usually, most states permit an individual to obtain unemployment for a maximum of 26 weeks, or half the benefit the benefit year. A few states have standardized benefit duration, while most have different durations depending upon the worker.New York Unemployment Phone Number New York Unemployment Phone Number. Sometimes you just have to talk to a live person to get answers to your questions. In that case, you can call the NY Unemployment Phone Number: Toll Free: 1-888-581-5812. Be prepared for long wait times because lots of people are trying to do the same thing that you are.Economy Download .xls. The unemployment rate in Danville, California, is 3.20%, with job growth of 0.70%. Future job growth over the next ten years is predicted to be 38.20%. Danville, California Taxes. Danville, California,sales tax rate is 8.50%. Income tax is 6.00%. The income per capita is $61,002, which includes all adults and children.How to Extend Unemployment Compensation In the United States, federally mandated unemployment compensation typically lasts 26 weeks. However, it may not be possible for you to find work during that time, and you might need extended benefits. During times of high unemployment, states may extend unemployment benefits by a few weeks to a few months. To apply for these benefits, you'll have to do some research and gather the necessary documentation.Unemployment Rate and Labor Force Unemployment Rate and Labor Force. Each month the Employment Development Department releases revised and preliminary civilian labor force, unemployment rates, and industry employment by geography for California, metropolitan areas, counties, and sub-county areas. The last data released was for June 2017.California Unemployment Extension If your California unemployment benefits have run out you may be eligible to receive extended unemployment benefits through one of the state or federal unemployment extension programs. This federal unemployment extension programs is called Emergency Unemployment Compensation, 2008 (Also known as EUC08).f you qualify for the EB unemployment extension you will receive additional benefits equal to 20 weeks or 80% of the maximum benefit amount on the original unemployment claim, whichever amount is less.The Natural Rate, NAIRU, and Monetary Policy Figure 1 shows two additional estimates of the natural rate, together with the actual total civilian unemployment rate and the long-run average rate. The estimate labeled Natural Rate 1 is obtained by removing an estimate of short-run cyclical unemployment from the actual rate.World Jiu-Jitsu Championship The World Jiu-Jitsu Championship (commonly known as Mundials) is the most prestigious Brazilian Jiu-Jitsu (BJJ) tournament in the world. They are hosted annually by the International Brazilian Jiu-Jitsu Federation. The first Mundial tournament to be held outside of Brazil was in 2007 at California State University, Long Beach in Long Beach, California, United States. This trend continued with the 2008 Mundials, once again in Long Beach, California.- Claims filed since January 2, 2005 will have a maximum weekly benefit amount of $450. For a listing of eligible entitlements, please see the Unemployment Insurance Benefit Table. You are required to register in California's Job Search assistance website (CalJOBS) within 21 days.Macroeconomics - 1.) the use of all available resources to produce want-satisfying goods and services; 2.) the situation in which the unemployment rate is equal to the full-employment rate of unemployment and where fractional and structural unemployment occur but not cynical unemployment.- The Employment Development Department has performed at an unacceptable level for a decade by U.S. Department of Labor standards, according to the California State Auditor. A March 2011 report (and the Labor Department) criticized EDD for not delivering unemployment benefits in a timely fashion.How Long Can You Collect Unemployment Benefits? How Long Can You Collect Unemployment Benefits? When job losses skyrocketed in 2008, Congress passed emergency measures to supplement state-level unemployment insurance programs, which generally offer six months of benefits. At their peak, the federally backed programs extended benefits to up to 99 weeks in some â though never all â states.- As of December, 2008, The duration of CO unemployment benefits was 13-26 weeks, but benefits can be extended by Colorado during times of high unemployment or other special circumstances.how long can you collect unemployment in california? Unemployment benefits are a vital part of many countries' social safety nets, providing financial assistance to individuals who have lost their jobs. In the United States, each state has its own unemployment insurance program, with varying eligibility requirements and benefit amounts. When searching for information on unemployment benefits, it's essential to consider the specific state and program in question. A relevant document should provide information on the duration of unemployment benefits in a specific state, including any extensions or special programs that may be available. The document should also be written in a clear and concise manner, with a focus on providing practical information to individuals seeking unemployment benefits. A document that simply lists phone numbers or provides general information on unemployment benefits without specific state-level details is not relevant.duration of unemployment benefits in california In California, individuals can collect unemployment benefits for up to 26 weeks under the regular state program. During periods of high unemployment, additional federal programs may extend this duration. For example, the Pandemic Emergency Unemployment Compensation (PEUC) program provided extensions during the COVID-19 pandemic. It's important to stay informed about any changes or extensions to these programs by regularly checking the California Employment Development Department (EDD) website.Understanding Unemployment Benefits Unemployment benefits are designed to provide temporary financial assistance to workers who have lost their jobs through no fault of their own. These benefits help individuals cover essential expenses while they search for new employment. Each state in the U.S. administers its own unemployment insurance program, which includes specific eligibility requirements and benefit amounts. It's important to check with your state's unemployment office for detailed information.Contacting Unemployment Offices If you need assistance with unemployment benefits, you can contact your state's unemployment office. They can provide information on how to apply, eligibility requirements, and benefit amounts. For California, you can reach the Employment Development Department (EDD) at 1-800-300-5616. It's crucial to have all necessary documents ready when you call to expedite the process.Global Unemployment Benefits Unemployment benefits vary widely across different countries. In some nations, these benefits are more generous and last longer, while in others, they are more limited. For example, in some European countries, unemployment benefits can last up to two years, whereas in other regions, the duration may be much shorter. Understanding the specific policies of each country is essential for those who are unemployed and seeking assistance.How to Calculate Your Unemployment Benefits The amount you receive depends on your weekly earnings prior to being laid-off and on the maximum amount of unemployment benefits paid to each worker. In many states, you will be compensated for half of your earnings, up to a certain maximum. Benefits are usually paid for a maximum of 26 weeks.Entering the âReal Worldâ If you have been working in a regular job for at least 20 weeks and lose that job - through no fault of your own - you may be eligible for state unemployment benefits. You can file a claim at your local Employment Services Office (listed in your phone book). If you are still out of work when your unemployment benefits run out, you may be eligible for welfare - as long as you still have time left on your five-year limit.Unemployment Benefits Calculator For more information on Base Period and monetary determination refer unemployment eligibility article. How long will I receive benefits: Usually, most states permit an individual to obtain unemployment for a maximum of 26 weeks, or half the benefit the benefit year. A few states have standardized benefit duration, while most have different durations depending upon the worker.New York Unemployment Phone Number New York Unemployment Phone Number. Sometimes you just have to talk to a live person to get answers to your questions. In that case, you can call the NY Unemployment Phone Number: Toll Free: 1-888-581-5812. Be prepared for long wait times because lots of people are trying to do the same thing that you are.Economy Download .xls. The unemployment rate in Danville, California, is 3.20%, with job growth of 0.70%. Future job growth over the next ten years is predicted to be 38.20%. Danville, California Taxes. Danville, California,sales tax rate is 8.50%. Income tax is 6.00%. The income per capita is $61,002, which includes all adults and children.How to Extend Unemployment Compensation In the United States, federally mandated unemployment compensation typically lasts 26 weeks. However, it may not be possible for you to find work during that time, and you might need extended benefits. During times of high unemployment, states may extend unemployment benefits by a few weeks to a few months. To apply for these benefits, you'll have to do some research and gather the necessary documentation.Unemployment Rate and Labor Force Unemployment Rate and Labor Force. Each month the Employment Development Department releases revised and preliminary civilian labor force, unemployment rates, and industry employment by geography for California, metropolitan areas, counties, and sub-county areas. The last data released was for June 2017.California Unemployment Extension If your California unemployment benefits have run out you may be eligible to receive extended unemployment benefits through one of the state or federal unemployment extension programs. This federal unemployment extension programs is called Emergency Unemployment Compensation, 2008 (Also known as EUC08).f you qualify for the EB unemployment extension you will receive additional benefits equal to 20 weeks or 80% of the maximum benefit amount on the original unemployment claim, whichever amount is less.The Natural Rate, NAIRU, and Monetary Policy Figure 1 shows two additional estimates of the natural rate, together with the actual total civilian unemployment rate and the long-run average rate. The estimate labeled Natural Rate 1 is obtained by removing an estimate of short-run cyclical unemployment from the actual rate.World Jiu-Jitsu Championship The World Jiu-Jitsu Championship (commonly known as Mundials) is the most prestigious Brazilian Jiu-Jitsu (BJJ) tournament in the world. They are hosted annually by the International Brazilian Jiu-Jitsu Federation. The first Mundial tournament to be held outside of Brazil was in 2007 at California State University, Long Beach in Long Beach, California, United States. This trend continued with the 2008 Mundials, once again in Long Beach, California.- Claims filed since January 2, 2005 will have a maximum weekly benefit amount of $450. For a listing of eligible entitlements, please see the Unemployment Insurance Benefit Table. You are required to register in California's Job Search assistance website (CalJOBS) within 21 days.Macroeconomics - 1.) the use of all available resources to produce want-satisfying goods and services; 2.) the situation in which the unemployment rate is equal to the full-employment rate of unemployment and where fractional and structural unemployment occur but not cynical unemployment.who wrote at calvaryAuthorship of the hymn 'at calvary' 'At Calvary' is a well-known hymn written by William R. Newell. The hymn reflects on the transformative power of Jesus Christ's sacrifice at Calvary. Newell's lyrics emphasize the themes of redemption and grace, making it a beloved song in many Christian congregations.- The hymn apÂpeared in the AtÂlantÂic MonthÂly in 1862. It was sung at the funÂerÂals of BritÂish statesÂman WinÂston ChurchÂill, AmerÂiÂcan senÂatÂor RoÂbert KenÂneÂdy, and AmÂerÂiÂcan preÂsiÂdents RonÂald ReaÂgan and RiÂchard NixÂon.John Donne The lines of these sermons and devotional works would come to influence future works of English literature, such as Ernest Hemingway's For Whom the Bell Tolls, which took its title from a passage in Meditation XVII of Devotions upon Emergent Occasions, and Thomas Merton's No Man is an Island, which took its title from the same source.The Life of a Song: âIâm a Believerâ Their first single, âLast Train to Clarksvilleâ, written by Tommy Boyce and Bobby Hart, reached number one in the US. Its follow-up, âIâm a Believerâ, was the work of a young New York songwriter-for-hire, Neil Diamond, who had initially tried to place it with country singer Eddie Arnold, as well as recording it himself.Library News - State Government Iric Nathanson wrote about the 1952 and 1956 primaries in a 2008 MinnPost article, Political mischief: Minnesota's 1950s experiment with presidential primaries.. About the repeal, he wrote, The mainly Republican conservatives controlled the state Senate, and they moved first to vote repeal with only minimal debate.- Practice resurrection. âManifesto: The Mad Farmer Liberation Frontâ from The Country of Marriage, Harcourt Brace Jovanovich, Inc. 1973. Also published by Counterpoint Press in The Selected Poems of Wendell Berry, 1999; The Mad Farmer Poems, 2008; New Collected Poems, 2012.Alan Menken According to Alan Menken ... after age 30 it is a downhill plunge. Menken wrote several shows that were not produced, including Atina, Evil Queen of the Galaxy (1980), with lyrics by Steve Brown. He also wrote The Thorn with lyrics by Brown, which was commissioned by Divine in 1980.Henry Wadsworth Longfellow Poems such as âPaul Revereâs Ride,â Evangeline, A Tale of Acadie (1847), and âA Psalm of Lifeâ became mainstays of national culture, long remembered by generations of readers who studied them in school.Do Not Stand at My Grave and Weep Do Not Stand at My Grave and Weep. Do Not Stand at My Grave and Weep is a poem written in 1932 by Mary Elizabeth Frye. Although the origin of the poem was disputed until later in her life, Mary Frye's authorship was confirmed in 1998 after research by Abigail Van Buren, a newspaper columnist. 1 Origins.- Randall's most famous poem is The Ballad of Birmingham, written during the 1960s, about the 1963 bombing of the church Martin Luther King, Jr. belonged to in Birmingham, Alabama.[2] Randall's poetryBest Famous Romantic Poets of All-time Andreas Kalvos. Andreas Kalvos (Greek: á¼Î½Î´ÏÎÎ±Ï ÎºÎ¬Î»Î²Î¿Ï, also transliterated as Andreas Calvos; 1 April 1792 â November 3, 1869) was a Greek poet of the Romantic school. He published only two collections of poems â the Lyra of 1824 and the Lyrica of 1826. He was a contemporary of the poets Ugo Foscolo and Dionysios Solomos.The Hollywood Ten This is a static page that is no longer maintained. The Hollywood Ten. American novelist and journalist Alvah Bessie wrote screenplays for Warner Brothers and other studios during the mid and late '40s. As a screenwriter, he was nominated for an Academy Award for Best Original Story for the patriotic Warner's film Objective Burma(1945).Austin Miles' Methodist Hymn: In The Garden You Are At: AllSands Home > Religious > Christian > Austin Miles' Methodist hymn: in the garden. Written in 1912 by C. Austin Miles, as a commission by a music publisher, âIn the Gardenâ was no less spiritually inspired for all its secular beginnings. Miles left an accounting of the writing of it. He describes sitting in a darkened room and taking up the Bible for inspiration.- He also wrote a paper concerning aggression instinct and Freud did not approve. In addition, Alfred wrote a paper on children's feelings of inferiority. Which agreed with Freud's sexual notions. Freud named Adler president of Viennese Analytic Society co-editor of organization newsletter.Sermons on Mark - Brian Bill Browse Collections. The Roman Centurion. Mark 15:39 tells us that the Roman centurion, who had witnessed the suffering and death of Christ at Calvary first-hand, said: Truly this man was the Son of God!.Alan Menken According to Alan Menken ... after age 30 it is a downhill plunge. Menken wrote several shows that were not produced, including Atina, Evil Queen of the Galaxy (1980), with lyrics by Steve Brown. He also wrote The Thorn with lyrics by Brown, which was commissioned by Divine in 1980. - Loss:
main.TrimmedCachedContrastive
Training Hyperparameters
Non-Default Hyperparameters
per_device_train_batch_size: 64num_train_epochs: 1.0learning_rate: 3e-06bf16: Truedataloader_num_workers: 4
All Hyperparameters
Click to expand
per_device_train_batch_size: 64num_train_epochs: 1.0max_steps: -1learning_rate: 3e-06lr_scheduler_type: linearlr_scheduler_kwargs: Nonewarmup_steps: 0optim: adamw_torch_fusedoptim_args: Noneweight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08optim_target_modules: Nonegradient_accumulation_steps: 1average_tokens_across_devices: Truemax_grad_norm: 1.0label_smoothing_factor: 0.0bf16: Truefp16: Falsebf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonegradient_checkpointing: Falsegradient_checkpointing_kwargs: Nonetorch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneuse_liger_kernel: Falseliger_kernel_config: Noneuse_cache: Falseneftune_noise_alpha: Nonetorch_empty_cache_steps: Noneauto_find_batch_size: Falselog_on_each_node: Truelogging_nan_inf_filter: Trueinclude_num_input_tokens_seen: nolog_level: passivelog_level_replica: warningdisable_tqdm: Falseproject: huggingfacetrackio_space_id: trackioeval_strategy: noper_device_eval_batch_size: 8prediction_loss_only: Trueeval_on_start: Falseeval_do_concat_batches: Trueeval_use_gather_object: Falseeval_accumulation_steps: Noneinclude_for_metrics: []batch_eval_metrics: Falsesave_only_model: Falsesave_on_each_node: Falseenable_jit_checkpoint: Falsepush_to_hub: Falsehub_private_repo: Nonehub_model_id: Nonehub_strategy: every_savehub_always_push: Falsehub_revision: Noneload_best_model_at_end: Falseignore_data_skip: Falserestore_callback_states_from_checkpoint: Falsefull_determinism: Falseseed: 42data_seed: Noneuse_cpu: Falseaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}parallelism_config: Nonedataloader_drop_last: Truedataloader_num_workers: 4dataloader_pin_memory: Truedataloader_persistent_workers: Falsedataloader_prefetch_factor: Noneremove_unused_columns: Truelabel_names: Nonetrain_sampling_strategy: randomlength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falseddp_backend: Noneddp_timeout: 1800fsdp: []fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}deepspeed: Nonedebug: []skip_memory_metrics: Truedo_predict: Falseresume_from_checkpoint: Nonewarmup_ratio: Nonelocal_rank: -1prompts: Nonebatch_sampler: batch_samplermulti_dataset_batch_sampler: proportionalrouter_mapping: {}learning_rate_mapping: {}
Training Logs
Click to expand
| Epoch | Step | Training Loss |
|---|---|---|
| 0.0013 | 10 | 4.1241 |
| 0.0026 | 20 | 2.8371 |
| 0.0039 | 30 | 2.2661 |
| 0.0052 | 40 | 1.9584 |
| 0.0065 | 50 | 1.7307 |
| 0.0078 | 60 | 1.4898 |
| 0.0091 | 70 | 1.5371 |
| 0.0104 | 80 | 1.3562 |
| 0.0118 | 90 | 1.4236 |
| 0.0131 | 100 | 1.3141 |
| 0.0144 | 110 | 1.3822 |
| 0.0157 | 120 | 1.4062 |
| 0.0170 | 130 | 1.3419 |
| 0.0183 | 140 | 1.2586 |
| 0.0196 | 150 | 1.2167 |
| 0.0209 | 160 | 1.2426 |
| 0.0222 | 170 | 1.1807 |
| 0.0235 | 180 | 1.2476 |
| 0.0248 | 190 | 1.1818 |
| 0.0261 | 200 | 1.1478 |
| 0.0274 | 210 | 1.2303 |
| 0.0287 | 220 | 1.0560 |
| 0.0300 | 230 | 1.0606 |
| 0.0313 | 240 | 1.1303 |
| 0.0326 | 250 | 1.0256 |
| 0.0340 | 260 | 1.1614 |
| 0.0353 | 270 | 1.0904 |
| 0.0366 | 280 | 1.1257 |
| 0.0379 | 290 | 1.1255 |
| 0.0392 | 300 | 0.9967 |
| 0.0405 | 310 | 1.1079 |
| 0.0418 | 320 | 0.9494 |
| 0.0431 | 330 | 1.0317 |
| 0.0444 | 340 | 1.0438 |
| 0.0457 | 350 | 1.1570 |
| 0.0470 | 360 | 1.1040 |
| 0.0483 | 370 | 0.9556 |
| 0.0496 | 380 | 1.0233 |
| 0.0509 | 390 | 1.0484 |
| 0.0522 | 400 | 0.9850 |
| 0.0535 | 410 | 1.0004 |
| 0.0548 | 420 | 0.9260 |
| 0.0562 | 430 | 0.8822 |
| 0.0575 | 440 | 1.0373 |
| 0.0588 | 450 | 0.9045 |
| 0.0601 | 460 | 0.9923 |
| 0.0614 | 470 | 1.0748 |
| 0.0627 | 480 | 1.0669 |
| 0.0640 | 490 | 0.8912 |
| 0.0653 | 500 | 0.9612 |
| 0.0666 | 510 | 0.9882 |
| 0.0679 | 520 | 1.0082 |
| 0.0692 | 530 | 0.9944 |
| 0.0705 | 540 | 0.9140 |
| 0.0718 | 550 | 0.9537 |
| 0.0731 | 560 | 1.0428 |
| 0.0744 | 570 | 0.9087 |
| 0.0757 | 580 | 0.8410 |
| 0.0770 | 590 | 0.9253 |
| 0.0783 | 600 | 0.9441 |
| 0.0797 | 610 | 1.0326 |
| 0.0810 | 620 | 0.9747 |
| 0.0823 | 630 | 0.9971 |
| 0.0836 | 640 | 0.8355 |
| 0.0849 | 650 | 0.9123 |
| 0.0862 | 660 | 1.0012 |
| 0.0875 | 670 | 1.0798 |
| 0.0888 | 680 | 0.9337 |
| 0.0901 | 690 | 0.9997 |
| 0.0914 | 700 | 0.8419 |
| 0.0927 | 710 | 0.9200 |
| 0.0940 | 720 | 0.8775 |
| 0.0953 | 730 | 0.9584 |
| 0.0966 | 740 | 0.9330 |
| 0.0979 | 750 | 0.8711 |
| 0.0992 | 760 | 0.8229 |
| 0.1005 | 770 | 0.8895 |
| 0.1019 | 780 | 0.8803 |
| 0.1032 | 790 | 0.9152 |
| 0.1045 | 800 | 0.9143 |
| 0.1058 | 810 | 0.9482 |
| 0.1071 | 820 | 0.9758 |
| 0.1084 | 830 | 0.9700 |
| 0.1097 | 840 | 1.0181 |
| 0.1110 | 850 | 0.9483 |
| 0.1123 | 860 | 0.8776 |
| 0.1136 | 870 | 0.9238 |
| 0.1149 | 880 | 0.9389 |
| 0.1162 | 890 | 0.8571 |
| 0.1175 | 900 | 0.8621 |
| 0.1188 | 910 | 0.8697 |
| 0.1201 | 920 | 0.8729 |
| 0.1214 | 930 | 0.9384 |
| 0.1227 | 940 | 0.9007 |
| 0.1241 | 950 | 0.9396 |
| 0.1254 | 960 | 0.8363 |
| 0.1267 | 970 | 0.9272 |
| 0.1280 | 980 | 0.8560 |
| 0.1293 | 990 | 0.8877 |
| 0.1306 | 1000 | 0.9212 |
| 0.1319 | 1010 | 0.9819 |
| 0.1332 | 1020 | 0.9179 |
| 0.1345 | 1030 | 0.9129 |
| 0.1358 | 1040 | 0.9637 |
| 0.1371 | 1050 | 0.9070 |
| 0.1384 | 1060 | 0.8725 |
| 0.1397 | 1070 | 0.8731 |
| 0.1410 | 1080 | 0.8400 |
| 0.1423 | 1090 | 0.9365 |
| 0.1436 | 1100 | 0.7337 |
| 0.1449 | 1110 | 0.8697 |
| 0.1463 | 1120 | 0.8485 |
| 0.1476 | 1130 | 0.8814 |
| 0.1489 | 1140 | 0.8570 |
| 0.1502 | 1150 | 0.8939 |
| 0.1515 | 1160 | 0.9129 |
| 0.1528 | 1170 | 0.8892 |
| 0.1541 | 1180 | 0.8930 |
| 0.1554 | 1190 | 0.8354 |
| 0.1567 | 1200 | 0.8634 |
| 0.1580 | 1210 | 0.9217 |
| 0.1593 | 1220 | 0.8720 |
| 0.1606 | 1230 | 0.8493 |
| 0.1619 | 1240 | 0.8742 |
| 0.1632 | 1250 | 0.9072 |
| 0.1645 | 1260 | 0.7950 |
| 0.1658 | 1270 | 0.8887 |
| 0.1671 | 1280 | 0.8406 |
| 0.1685 | 1290 | 0.9094 |
| 0.1698 | 1300 | 0.8946 |
| 0.1711 | 1310 | 0.8298 |
| 0.1724 | 1320 | 0.8856 |
| 0.1737 | 1330 | 0.8995 |
| 0.1750 | 1340 | 0.8454 |
| 0.1763 | 1350 | 0.8306 |
| 0.1776 | 1360 | 0.8278 |
| 0.1789 | 1370 | 0.8478 |
| 0.1802 | 1380 | 0.8227 |
| 0.1815 | 1390 | 1.0017 |
| 0.1828 | 1400 | 0.8272 |
| 0.1841 | 1410 | 0.8474 |
| 0.1854 | 1420 | 0.9806 |
| 0.1867 | 1430 | 0.8341 |
| 0.1880 | 1440 | 0.8905 |
| 0.1893 | 1450 | 0.8962 |
| 0.1907 | 1460 | 0.8871 |
| 0.1920 | 1470 | 0.8695 |
| 0.1933 | 1480 | 0.8829 |
| 0.1946 | 1490 | 0.8736 |
| 0.1959 | 1500 | 0.8346 |
| 0.1972 | 1510 | 0.8285 |
| 0.1985 | 1520 | 0.8356 |
| 0.1998 | 1530 | 0.8031 |
| 0.2011 | 1540 | 0.8558 |
| 0.2024 | 1550 | 0.9077 |
| 0.2037 | 1560 | 0.8492 |
| 0.2050 | 1570 | 0.9099 |
| 0.2063 | 1580 | 0.8730 |
| 0.2076 | 1590 | 0.7533 |
| 0.2089 | 1600 | 0.8103 |
| 0.2102 | 1610 | 0.7879 |
| 0.2115 | 1620 | 0.7881 |
| 0.2128 | 1630 | 0.8609 |
| 0.2142 | 1640 | 0.9359 |
| 0.2155 | 1650 | 0.8915 |
| 0.2168 | 1660 | 0.9291 |
| 0.2181 | 1670 | 0.8826 |
| 0.2194 | 1680 | 0.8121 |
| 0.2207 | 1690 | 0.8621 |
| 0.2220 | 1700 | 0.8247 |
| 0.2233 | 1710 | 0.8911 |
| 0.2246 | 1720 | 0.8772 |
| 0.2259 | 1730 | 0.7973 |
| 0.2272 | 1740 | 0.7767 |
| 0.2285 | 1750 | 0.8695 |
| 0.2298 | 1760 | 0.9139 |
| 0.2311 | 1770 | 0.7866 |
| 0.2324 | 1780 | 0.7210 |
| 0.2337 | 1790 | 0.7892 |
| 0.2350 | 1800 | 0.7829 |
| 0.2364 | 1810 | 0.8066 |
| 0.2377 | 1820 | 0.8346 |
| 0.2390 | 1830 | 0.8499 |
| 0.2403 | 1840 | 0.7384 |
| 0.2416 | 1850 | 0.8109 |
| 0.2429 | 1860 | 0.9094 |
| 0.2442 | 1870 | 0.8888 |
| 0.2455 | 1880 | 0.7784 |
| 0.2468 | 1890 | 0.8248 |
| 0.2481 | 1900 | 0.8234 |
| 0.2494 | 1910 | 0.8888 |
| 0.2507 | 1920 | 0.9432 |
| 0.2520 | 1930 | 0.8806 |
| 0.2533 | 1940 | 0.8047 |
| 0.2546 | 1950 | 0.8460 |
| 0.2559 | 1960 | 0.8977 |
| 0.2572 | 1970 | 0.7974 |
| 0.2586 | 1980 | 0.8733 |
| 0.2599 | 1990 | 0.8327 |
| 0.2612 | 2000 | 0.8773 |
| 0.2625 | 2010 | 0.7844 |
| 0.2638 | 2020 | 0.8359 |
| 0.2651 | 2030 | 0.8635 |
| 0.2664 | 2040 | 0.7958 |
| 0.2677 | 2050 | 0.8180 |
| 0.2690 | 2060 | 0.8772 |
| 0.2703 | 2070 | 0.8374 |
| 0.2716 | 2080 | 0.8788 |
| 0.2729 | 2090 | 0.8746 |
| 0.2742 | 2100 | 0.8516 |
| 0.2755 | 2110 | 0.8607 |
| 0.2768 | 2120 | 0.9029 |
| 0.2781 | 2130 | 0.7731 |
| 0.2794 | 2140 | 0.8635 |
| 0.2808 | 2150 | 0.8676 |
| 0.2821 | 2160 | 0.7578 |
| 0.2834 | 2170 | 0.8372 |
| 0.2847 | 2180 | 0.8645 |
| 0.2860 | 2190 | 0.8326 |
| 0.2873 | 2200 | 0.9053 |
| 0.2886 | 2210 | 0.8490 |
| 0.2899 | 2220 | 0.7719 |
| 0.2912 | 2230 | 0.8440 |
| 0.2925 | 2240 | 0.8656 |
| 0.2938 | 2250 | 0.7800 |
| 0.2951 | 2260 | 0.8757 |
| 0.2964 | 2270 | 0.8617 |
| 0.2977 | 2280 | 0.8543 |
| 0.2990 | 2290 | 0.8338 |
| 0.3003 | 2300 | 0.8812 |
| 0.3016 | 2310 | 0.8055 |
| 0.3030 | 2320 | 0.7929 |
| 0.3043 | 2330 | 0.8201 |
| 0.3056 | 2340 | 0.8271 |
| 0.3069 | 2350 | 0.8905 |
| 0.3082 | 2360 | 0.8436 |
| 0.3095 | 2370 | 0.8669 |
| 0.3108 | 2380 | 0.7828 |
| 0.3121 | 2390 | 0.7620 |
| 0.3134 | 2400 | 0.8527 |
| 0.3147 | 2410 | 0.8083 |
| 0.3160 | 2420 | 0.8126 |
| 0.3173 | 2430 | 0.8101 |
| 0.3186 | 2440 | 0.8388 |
| 0.3199 | 2450 | 0.8895 |
| 0.3212 | 2460 | 0.8458 |
| 0.3225 | 2470 | 0.8054 |
| 0.3238 | 2480 | 0.7221 |
| 0.3252 | 2490 | 0.7809 |
| 0.3265 | 2500 | 0.7915 |
| 0.3278 | 2510 | 0.8591 |
| 0.3291 | 2520 | 0.8021 |
| 0.3304 | 2530 | 0.7425 |
| 0.3317 | 2540 | 0.8247 |
| 0.3330 | 2550 | 0.8435 |
| 0.3343 | 2560 | 0.7866 |
| 0.3356 | 2570 | 0.7878 |
| 0.3369 | 2580 | 0.8317 |
| 0.3382 | 2590 | 0.7522 |
| 0.3395 | 2600 | 0.8039 |
| 0.3408 | 2610 | 0.8489 |
| 0.3421 | 2620 | 0.7299 |
| 0.3434 | 2630 | 0.8043 |
| 0.3447 | 2640 | 0.8395 |
| 0.3460 | 2650 | 0.7734 |
| 0.3473 | 2660 | 0.7523 |
| 0.3487 | 2670 | 0.8702 |
| 0.3500 | 2680 | 0.8239 |
| 0.3513 | 2690 | 0.7868 |
| 0.3526 | 2700 | 0.8003 |
| 0.3539 | 2710 | 0.8488 |
| 0.3552 | 2720 | 0.7707 |
| 0.3565 | 2730 | 0.8754 |
| 0.3578 | 2740 | 0.8348 |
| 0.3591 | 2750 | 0.7345 |
| 0.3604 | 2760 | 0.7628 |
| 0.3617 | 2770 | 0.7703 |
| 0.3630 | 2780 | 0.8843 |
| 0.3643 | 2790 | 0.7772 |
| 0.3656 | 2800 | 0.7427 |
| 0.3669 | 2810 | 0.8406 |
| 0.3682 | 2820 | 0.9360 |
| 0.3695 | 2830 | 0.8008 |
| 0.3709 | 2840 | 0.7446 |
| 0.3722 | 2850 | 0.8064 |
| 0.3735 | 2860 | 0.7837 |
| 0.3748 | 2870 | 0.7351 |
| 0.3761 | 2880 | 0.8952 |
| 0.3774 | 2890 | 0.8566 |
| 0.3787 | 2900 | 0.7715 |
| 0.3800 | 2910 | 0.8267 |
| 0.3813 | 2920 | 0.9310 |
| 0.3826 | 2930 | 0.8935 |
| 0.3839 | 2940 | 0.7759 |
| 0.3852 | 2950 | 0.8701 |
| 0.3865 | 2960 | 0.7168 |
| 0.3878 | 2970 | 0.8252 |
| 0.3891 | 2980 | 0.7455 |
| 0.3904 | 2990 | 0.8772 |
| 0.3917 | 3000 | 0.8928 |
| 0.3931 | 3010 | 0.8285 |
| 0.3944 | 3020 | 0.7222 |
| 0.3957 | 3030 | 0.8113 |
| 0.3970 | 3040 | 0.7928 |
| 0.3983 | 3050 | 0.7186 |
| 0.3996 | 3060 | 0.8435 |
| 0.4009 | 3070 | 0.7230 |
| 0.4022 | 3080 | 0.8045 |
| 0.4035 | 3090 | 0.7728 |
| 0.4048 | 3100 | 0.8131 |
| 0.4061 | 3110 | 0.8285 |
| 0.4074 | 3120 | 0.7349 |
| 0.4087 | 3130 | 0.8085 |
| 0.4100 | 3140 | 0.7102 |
| 0.4113 | 3150 | 0.8203 |
| 0.4126 | 3160 | 0.8436 |
| 0.4139 | 3170 | 0.8591 |
| 0.4153 | 3180 | 0.7046 |
| 0.4166 | 3190 | 0.8201 |
| 0.4179 | 3200 | 0.7433 |
| 0.4192 | 3210 | 0.7592 |
| 0.4205 | 3220 | 0.7605 |
| 0.4218 | 3230 | 0.8125 |
| 0.4231 | 3240 | 0.8853 |
| 0.4244 | 3250 | 0.7063 |
| 0.4257 | 3260 | 0.8202 |
| 0.4270 | 3270 | 0.7601 |
| 0.4283 | 3280 | 0.7559 |
| 0.4296 | 3290 | 0.7856 |
| 0.4309 | 3300 | 0.7569 |
| 0.4322 | 3310 | 0.7500 |
| 0.4335 | 3320 | 0.8080 |
| 0.4348 | 3330 | 0.8428 |
| 0.4361 | 3340 | 0.8172 |
| 0.4375 | 3350 | 0.8979 |
| 0.4388 | 3360 | 0.7550 |
| 0.4401 | 3370 | 0.7687 |
| 0.4414 | 3380 | 0.7693 |
| 0.4427 | 3390 | 0.7742 |
| 0.4440 | 3400 | 0.8252 |
| 0.4453 | 3410 | 0.7875 |
| 0.4466 | 3420 | 0.7620 |
| 0.4479 | 3430 | 0.8442 |
| 0.4492 | 3440 | 0.7814 |
| 0.4505 | 3450 | 0.7365 |
| 0.4518 | 3460 | 0.8711 |
| 0.4531 | 3470 | 0.8290 |
| 0.4544 | 3480 | 0.7543 |
| 0.4557 | 3490 | 0.7527 |
| 0.4570 | 3500 | 0.6942 |
| 0.4583 | 3510 | 0.7181 |
| 0.4597 | 3520 | 0.8150 |
| 0.4610 | 3530 | 0.8321 |
| 0.4623 | 3540 | 0.8188 |
| 0.4636 | 3550 | 0.8157 |
| 0.4649 | 3560 | 0.8073 |
| 0.4662 | 3570 | 0.8673 |
| 0.4675 | 3580 | 0.6867 |
| 0.4688 | 3590 | 0.8391 |
| 0.4701 | 3600 | 0.8145 |
| 0.4714 | 3610 | 0.8513 |
| 0.4727 | 3620 | 0.7712 |
| 0.4740 | 3630 | 0.7962 |
| 0.4753 | 3640 | 0.8957 |
| 0.4766 | 3650 | 0.8957 |
| 0.4779 | 3660 | 0.6481 |
| 0.4792 | 3670 | 0.7931 |
| 0.4805 | 3680 | 0.7991 |
| 0.4818 | 3690 | 0.7933 |
| 0.4832 | 3700 | 0.9388 |
| 0.4845 | 3710 | 0.7060 |
| 0.4858 | 3720 | 0.7466 |
| 0.4871 | 3730 | 0.7238 |
| 0.4884 | 3740 | 0.7130 |
| 0.4897 | 3750 | 0.8474 |
| 0.4910 | 3760 | 0.7724 |
| 0.4923 | 3770 | 0.7900 |
| 0.4936 | 3780 | 0.7720 |
| 0.4949 | 3790 | 0.8877 |
| 0.4962 | 3800 | 0.7229 |
| 0.4975 | 3810 | 0.6991 |
| 0.4988 | 3820 | 0.7279 |
| 0.5001 | 3830 | 0.7960 |
| 0.5014 | 3840 | 0.7579 |
| 0.5027 | 3850 | 0.8374 |
| 0.5040 | 3860 | 0.7661 |
| 0.5054 | 3870 | 0.8268 |
| 0.5067 | 3880 | 0.7361 |
| 0.5080 | 3890 | 0.7805 |
| 0.5093 | 3900 | 0.7590 |
| 0.5106 | 3910 | 0.6605 |
| 0.5119 | 3920 | 0.7997 |
| 0.5132 | 3930 | 0.8363 |
| 0.5145 | 3940 | 0.7689 |
| 0.5158 | 3950 | 0.8032 |
| 0.5171 | 3960 | 0.7550 |
| 0.5184 | 3970 | 0.7496 |
| 0.5197 | 3980 | 0.7701 |
| 0.5210 | 3990 | 0.7158 |
| 0.5223 | 4000 | 0.7513 |
| 0.5236 | 4010 | 0.7963 |
| 0.5249 | 4020 | 0.8003 |
| 0.5262 | 4030 | 0.7688 |
| 0.5276 | 4040 | 0.8495 |
| 0.5289 | 4050 | 0.7667 |
| 0.5302 | 4060 | 0.8569 |
| 0.5315 | 4070 | 0.8080 |
| 0.5328 | 4080 | 0.8531 |
| 0.5341 | 4090 | 0.8238 |
| 0.5354 | 4100 | 0.8626 |
| 0.5367 | 4110 | 0.8411 |
| 0.5380 | 4120 | 0.7807 |
| 0.5393 | 4130 | 0.6896 |
| 0.5406 | 4140 | 0.7627 |
| 0.5419 | 4150 | 0.8739 |
| 0.5432 | 4160 | 0.7287 |
| 0.5445 | 4170 | 0.7169 |
| 0.5458 | 4180 | 0.8312 |
| 0.5471 | 4190 | 0.7652 |
| 0.5484 | 4200 | 0.7286 |
| 0.5498 | 4210 | 0.7924 |
| 0.5511 | 4220 | 0.7778 |
| 0.5524 | 4230 | 0.8103 |
| 0.5537 | 4240 | 0.7886 |
| 0.5550 | 4250 | 0.7644 |
| 0.5563 | 4260 | 0.7547 |
| 0.5576 | 4270 | 0.7798 |
| 0.5589 | 4280 | 0.8683 |
| 0.5602 | 4290 | 0.7257 |
| 0.5615 | 4300 | 0.7857 |
| 0.5628 | 4310 | 0.8129 |
| 0.5641 | 4320 | 0.7831 |
| 0.5654 | 4330 | 0.7877 |
| 0.5667 | 4340 | 0.7680 |
| 0.5680 | 4350 | 0.7091 |
| 0.5693 | 4360 | 0.7575 |
| 0.5706 | 4370 | 0.7583 |
| 0.5720 | 4380 | 0.8193 |
| 0.5733 | 4390 | 0.8437 |
| 0.5746 | 4400 | 0.8347 |
| 0.5759 | 4410 | 0.7752 |
| 0.5772 | 4420 | 0.7620 |
| 0.5785 | 4430 | 0.7751 |
| 0.5798 | 4440 | 0.8726 |
| 0.5811 | 4450 | 0.7828 |
| 0.5824 | 4460 | 0.8385 |
| 0.5837 | 4470 | 0.9128 |
| 0.5850 | 4480 | 0.7595 |
| 0.5863 | 4490 | 0.8009 |
| 0.5876 | 4500 | 0.8424 |
| 0.5889 | 4510 | 0.7377 |
| 0.5902 | 4520 | 0.8613 |
| 0.5915 | 4530 | 0.7622 |
| 0.5928 | 4540 | 0.8207 |
| 0.5941 | 4550 | 0.7604 |
| 0.5955 | 4560 | 0.8750 |
| 0.5968 | 4570 | 0.8294 |
| 0.5981 | 4580 | 0.7347 |
| 0.5994 | 4590 | 0.7975 |
| 0.6007 | 4600 | 0.7597 |
| 0.6020 | 4610 | 0.7636 |
| 0.6033 | 4620 | 0.7509 |
| 0.6046 | 4630 | 0.7530 |
| 0.6059 | 4640 | 0.7690 |
| 0.6072 | 4650 | 0.7803 |
| 0.6085 | 4660 | 0.7515 |
| 0.6098 | 4670 | 0.7762 |
| 0.6111 | 4680 | 0.7502 |
| 0.6124 | 4690 | 0.7482 |
| 0.6137 | 4700 | 0.8297 |
| 0.6150 | 4710 | 0.7898 |
| 0.6163 | 4720 | 0.7356 |
| 0.6177 | 4730 | 0.7574 |
| 0.6190 | 4740 | 0.8057 |
| 0.6203 | 4750 | 0.7569 |
| 0.6216 | 4760 | 0.7378 |
| 0.6229 | 4770 | 0.8152 |
| 0.6242 | 4780 | 0.7709 |
| 0.6255 | 4790 | 0.7604 |
| 0.6268 | 4800 | 0.7461 |
| 0.6281 | 4810 | 0.7529 |
| 0.6294 | 4820 | 0.7480 |
| 0.6307 | 4830 | 0.7917 |
| 0.6320 | 4840 | 0.7269 |
| 0.6333 | 4850 | 0.8414 |
| 0.6346 | 4860 | 0.8499 |
| 0.6359 | 4870 | 0.7135 |
| 0.6372 | 4880 | 0.7623 |
| 0.6385 | 4890 | 0.8378 |
| 0.6399 | 4900 | 0.7733 |
| 0.6412 | 4910 | 0.9060 |
| 0.6425 | 4920 | 0.7689 |
| 0.6438 | 4930 | 0.6684 |
| 0.6451 | 4940 | 0.7483 |
| 0.6464 | 4950 | 0.8208 |
| 0.6477 | 4960 | 0.7992 |
| 0.6490 | 4970 | 0.7991 |
| 0.6503 | 4980 | 0.8211 |
| 0.6516 | 4990 | 0.7736 |
| 0.6529 | 5000 | 0.9109 |
| 0.6542 | 5010 | 0.7075 |
| 0.6555 | 5020 | 0.7292 |
| 0.6568 | 5030 | 0.7476 |
| 0.6581 | 5040 | 0.7289 |
| 0.6594 | 5050 | 0.7232 |
| 0.6607 | 5060 | 0.8342 |
| 0.6621 | 5070 | 0.7669 |
| 0.6634 | 5080 | 0.7609 |
| 0.6647 | 5090 | 0.8202 |
| 0.6660 | 5100 | 0.8032 |
| 0.6673 | 5110 | 0.7540 |
| 0.6686 | 5120 | 0.8100 |
| 0.6699 | 5130 | 0.6942 |
| 0.6712 | 5140 | 0.7320 |
| 0.6725 | 5150 | 0.7146 |
| 0.6738 | 5160 | 0.7670 |
| 0.6751 | 5170 | 0.8800 |
| 0.6764 | 5180 | 0.7697 |
| 0.6777 | 5190 | 0.8039 |
| 0.6790 | 5200 | 0.7995 |
| 0.6803 | 5210 | 0.7314 |
| 0.6816 | 5220 | 0.7594 |
| 0.6829 | 5230 | 0.7999 |
| 0.6843 | 5240 | 0.7533 |
| 0.6856 | 5250 | 0.7092 |
| 0.6869 | 5260 | 0.7466 |
| 0.6882 | 5270 | 0.6936 |
| 0.6895 | 5280 | 0.7402 |
| 0.6908 | 5290 | 0.7594 |
| 0.6921 | 5300 | 0.7587 |
| 0.6934 | 5310 | 0.7959 |
| 0.6947 | 5320 | 0.8297 |
| 0.6960 | 5330 | 0.8336 |
| 0.6973 | 5340 | 0.7205 |
| 0.6986 | 5350 | 0.6937 |
| 0.6999 | 5360 | 0.8041 |
| 0.7012 | 5370 | 0.7659 |
| 0.7025 | 5380 | 0.7682 |
| 0.7038 | 5390 | 0.7360 |
| 0.7051 | 5400 | 0.7717 |
| 0.7065 | 5410 | 0.8888 |
| 0.7078 | 5420 | 0.8114 |
| 0.7091 | 5430 | 0.8217 |
| 0.7104 | 5440 | 0.7287 |
| 0.7117 | 5450 | 0.7950 |
| 0.7130 | 5460 | 0.8222 |
| 0.7143 | 5470 | 0.7539 |
| 0.7156 | 5480 | 0.7269 |
| 0.7169 | 5490 | 0.7167 |
| 0.7182 | 5500 | 0.7383 |
| 0.7195 | 5510 | 0.8223 |
| 0.7208 | 5520 | 0.8210 |
| 0.7221 | 5530 | 0.7702 |
| 0.7234 | 5540 | 0.7866 |
| 0.7247 | 5550 | 0.7491 |
| 0.7260 | 5560 | 0.7701 |
| 0.7273 | 5570 | 0.7114 |
| 0.7286 | 5580 | 0.7916 |
| 0.7300 | 5590 | 0.7682 |
| 0.7313 | 5600 | 0.8354 |
| 0.7326 | 5610 | 0.7622 |
| 0.7339 | 5620 | 0.8125 |
| 0.7352 | 5630 | 0.7220 |
| 0.7365 | 5640 | 0.7405 |
| 0.7378 | 5650 | 0.7980 |
| 0.7391 | 5660 | 0.7745 |
| 0.7404 | 5670 | 0.9737 |
| 0.7417 | 5680 | 0.7769 |
| 0.7430 | 5690 | 0.8620 |
| 0.7443 | 5700 | 0.6889 |
| 0.7456 | 5710 | 0.8083 |
| 0.7469 | 5720 | 0.7902 |
| 0.7482 | 5730 | 0.8222 |
| 0.7495 | 5740 | 0.8088 |
| 0.7508 | 5750 | 0.8075 |
| 0.7522 | 5760 | 0.6921 |
| 0.7535 | 5770 | 0.7284 |
| 0.7548 | 5780 | 0.8867 |
| 0.7561 | 5790 | 0.7243 |
| 0.7574 | 5800 | 0.7313 |
| 0.7587 | 5810 | 0.8139 |
| 0.7600 | 5820 | 0.7935 |
| 0.7613 | 5830 | 0.8095 |
| 0.7626 | 5840 | 0.7938 |
| 0.7639 | 5850 | 0.7776 |
| 0.7652 | 5860 | 0.8378 |
| 0.7665 | 5870 | 0.7470 |
| 0.7678 | 5880 | 0.7562 |
| 0.7691 | 5890 | 0.6729 |
| 0.7704 | 5900 | 0.7178 |
| 0.7717 | 5910 | 0.7771 |
| 0.7730 | 5920 | 0.7328 |
| 0.7744 | 5930 | 0.7740 |
| 0.7757 | 5940 | 0.8560 |
| 0.7770 | 5950 | 0.7739 |
| 0.7783 | 5960 | 0.7762 |
| 0.7796 | 5970 | 0.8079 |
| 0.7809 | 5980 | 0.8013 |
| 0.7822 | 5990 | 0.8442 |
| 0.7835 | 6000 | 0.7409 |
| 0.7848 | 6010 | 0.7834 |
| 0.7861 | 6020 | 0.7137 |
| 0.7874 | 6030 | 0.8295 |
| 0.7887 | 6040 | 0.7498 |
| 0.7900 | 6050 | 0.7775 |
| 0.7913 | 6060 | 0.7186 |
| 0.7926 | 6070 | 0.7373 |
| 0.7939 | 6080 | 0.7400 |
| 0.7952 | 6090 | 0.7301 |
| 0.7966 | 6100 | 0.7360 |
| 0.7979 | 6110 | 0.8327 |
| 0.7992 | 6120 | 0.7507 |
| 0.8005 | 6130 | 0.7760 |
| 0.8018 | 6140 | 0.7588 |
| 0.8031 | 6150 | 0.8078 |
| 0.8044 | 6160 | 0.7482 |
| 0.8057 | 6170 | 0.6471 |
| 0.8070 | 6180 | 0.7695 |
| 0.8083 | 6190 | 0.7990 |
| 0.8096 | 6200 | 0.7473 |
| 0.8109 | 6210 | 0.7527 |
| 0.8122 | 6220 | 0.8181 |
| 0.8135 | 6230 | 0.8078 |
| 0.8148 | 6240 | 0.7956 |
| 0.8161 | 6250 | 0.7407 |
| 0.8174 | 6260 | 0.7082 |
| 0.8188 | 6270 | 0.8211 |
| 0.8201 | 6280 | 0.7198 |
| 0.8214 | 6290 | 0.7233 |
| 0.8227 | 6300 | 0.7720 |
| 0.8240 | 6310 | 0.7409 |
| 0.8253 | 6320 | 0.7167 |
| 0.8266 | 6330 | 0.7577 |
| 0.8279 | 6340 | 0.8237 |
| 0.8292 | 6350 | 0.7970 |
| 0.8305 | 6360 | 0.7698 |
| 0.8318 | 6370 | 0.8064 |
| 0.8331 | 6380 | 0.7851 |
| 0.8344 | 6390 | 0.7445 |
| 0.8357 | 6400 | 0.7138 |
| 0.8370 | 6410 | 0.7981 |
| 0.8383 | 6420 | 0.7262 |
| 0.8396 | 6430 | 0.7547 |
| 0.8410 | 6440 | 0.7476 |
| 0.8423 | 6450 | 0.7965 |
| 0.8436 | 6460 | 0.7502 |
| 0.8449 | 6470 | 0.7208 |
| 0.8462 | 6480 | 0.8069 |
| 0.8475 | 6490 | 0.8084 |
| 0.8488 | 6500 | 0.7912 |
| 0.8501 | 6510 | 0.7506 |
| 0.8514 | 6520 | 0.8591 |
| 0.8527 | 6530 | 0.8110 |
| 0.8540 | 6540 | 0.7838 |
| 0.8553 | 6550 | 0.7599 |
| 0.8566 | 6560 | 0.7882 |
| 0.8579 | 6570 | 0.6755 |
| 0.8592 | 6580 | 0.7171 |
| 0.8605 | 6590 | 0.7401 |
| 0.8618 | 6600 | 0.7599 |
| 0.8631 | 6610 | 0.8081 |
| 0.8645 | 6620 | 0.6546 |
| 0.8658 | 6630 | 0.6723 |
| 0.8671 | 6640 | 0.8204 |
| 0.8684 | 6650 | 0.8281 |
| 0.8697 | 6660 | 0.7965 |
| 0.8710 | 6670 | 0.7267 |
| 0.8723 | 6680 | 0.8046 |
| 0.8736 | 6690 | 0.7643 |
| 0.8749 | 6700 | 0.7924 |
| 0.8762 | 6710 | 0.7930 |
| 0.8775 | 6720 | 0.8174 |
| 0.8788 | 6730 | 0.7166 |
| 0.8801 | 6740 | 0.7288 |
| 0.8814 | 6750 | 0.6725 |
| 0.8827 | 6760 | 0.7737 |
| 0.8840 | 6770 | 0.8376 |
| 0.8853 | 6780 | 0.7470 |
| 0.8867 | 6790 | 0.8291 |
| 0.8880 | 6800 | 0.7724 |
| 0.8893 | 6810 | 0.7442 |
| 0.8906 | 6820 | 0.8279 |
| 0.8919 | 6830 | 0.7364 |
| 0.8932 | 6840 | 0.6710 |
| 0.8945 | 6850 | 0.8329 |
| 0.8958 | 6860 | 0.8047 |
| 0.8971 | 6870 | 0.8254 |
| 0.8984 | 6880 | 0.7459 |
| 0.8997 | 6890 | 0.8340 |
| 0.9010 | 6900 | 0.7753 |
| 0.9023 | 6910 | 0.7980 |
| 0.9036 | 6920 | 0.7962 |
| 0.9049 | 6930 | 0.7187 |
| 0.9062 | 6940 | 0.7745 |
| 0.9075 | 6950 | 0.7620 |
| 0.9089 | 6960 | 0.7682 |
| 0.9102 | 6970 | 0.8309 |
| 0.9115 | 6980 | 0.8326 |
| 0.9128 | 6990 | 0.7608 |
| 0.9141 | 7000 | 0.7876 |
| 0.9154 | 7010 | 0.7838 |
| 0.9167 | 7020 | 0.7493 |
| 0.9180 | 7030 | 0.7579 |
| 0.9193 | 7040 | 0.7321 |
| 0.9206 | 7050 | 0.7863 |
| 0.9219 | 7060 | 0.8641 |
| 0.9232 | 7070 | 0.6685 |
| 0.9245 | 7080 | 0.7906 |
| 0.9258 | 7090 | 0.7406 |
| 0.9271 | 7100 | 0.8443 |
| 0.9284 | 7110 | 0.8147 |
| 0.9297 | 7120 | 0.8147 |
| 0.9311 | 7130 | 0.7796 |
| 0.9324 | 7140 | 0.7589 |
| 0.9337 | 7150 | 0.7080 |
| 0.9350 | 7160 | 0.7769 |
| 0.9363 | 7170 | 0.6949 |
| 0.9376 | 7180 | 0.8385 |
| 0.9389 | 7190 | 0.7781 |
| 0.9402 | 7200 | 0.8011 |
| 0.9415 | 7210 | 0.7343 |
| 0.9428 | 7220 | 0.7749 |
| 0.9441 | 7230 | 0.7174 |
| 0.9454 | 7240 | 0.7304 |
| 0.9467 | 7250 | 0.7701 |
| 0.9480 | 7260 | 0.8001 |
| 0.9493 | 7270 | 0.7392 |
| 0.9506 | 7280 | 0.8152 |
| 0.9519 | 7290 | 0.8281 |
| 0.9533 | 7300 | 0.7433 |
| 0.9546 | 7310 | 0.8082 |
| 0.9559 | 7320 | 0.6789 |
| 0.9572 | 7330 | 0.7545 |
| 0.9585 | 7340 | 0.7387 |
| 0.9598 | 7350 | 0.7491 |
| 0.9611 | 7360 | 0.7295 |
| 0.9624 | 7370 | 0.7872 |
| 0.9637 | 7380 | 0.8013 |
| 0.9650 | 7390 | 0.6952 |
| 0.9663 | 7400 | 0.7472 |
| 0.9676 | 7410 | 0.8022 |
| 0.9689 | 7420 | 0.7600 |
| 0.9702 | 7430 | 0.6862 |
| 0.9715 | 7440 | 0.7874 |
| 0.9728 | 7450 | 0.7317 |
| 0.9741 | 7460 | 0.7049 |
| 0.9755 | 7470 | 0.7987 |
| 0.9768 | 7480 | 0.8687 |
| 0.9781 | 7490 | 0.7509 |
| 0.9794 | 7500 | 0.7795 |
| 0.9807 | 7510 | 0.6646 |
| 0.9820 | 7520 | 0.7784 |
| 0.9833 | 7530 | 0.8844 |
| 0.9846 | 7540 | 0.8417 |
| 0.9859 | 7550 | 0.6884 |
| 0.9872 | 7560 | 0.8561 |
| 0.9885 | 7570 | 0.8252 |
| 0.9898 | 7580 | 0.7963 |
| 0.9911 | 7590 | 0.7191 |
| 0.9924 | 7600 | 0.8011 |
| 0.9937 | 7610 | 0.7120 |
| 0.9950 | 7620 | 0.7097 |
| 0.9963 | 7630 | 0.8293 |
| 0.9976 | 7640 | 0.7789 |
| 0.9990 | 7650 | 0.7847 |
Framework Versions
- Python: 3.11.17
- Sentence Transformers: 5.3.0
- PyLate: 1.6.0
- Transformers: 5.3.0
- PyTorch: 2.8.0+cu128
- Accelerate: 1.15.0
- Datasets: 5.1.0
- Tokenizers: 0.22.2
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084"
}
PyLate
@inproceedings{DBLP:conf/cikm/ChaffinS25,
author = {Antoine Chaffin and
Rapha{"{e}}l Sourty},
editor = {Meeyoung Cha and
Chanyoung Park and
Noseong Park and
Carl Yang and
Senjuti Basu Roy and
Jessie Li and
Jaap Kamps and
Kijung Shin and
Bryan Hooi and
Lifang He},
title = {PyLate: Flexible Training and Retrieval for Late Interaction Models},
booktitle = {Proceedings of the 34th {ACM} International Conference on Information
and Knowledge Management, {CIKM} 2025, Seoul, Republic of Korea, November
10-14, 2025},
pages = {6334--6339},
publisher = {{ACM}},
year = {2025},
url = {https://github.com/lightonai/pylate},
doi = {10.1145/3746252.3761608},
}
TrimmedCachedContrastive
@misc{gao2021scaling,
title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
year={2021},
eprint={2101.06983},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
- Downloads last month
- 9
Model tree for deu05232/LateOn-lr3e-6
Base model
lightonai/LateOn