mindXtrain39 — generation 39 of the mindX dream→weights lineage (imprint Δ +0.1002, accepted)

Modelfile

@@ -0,0 +1,6 @@
+# ollama create mindXtrain39 -f Modelfile   (run it from this repo's directory)
+FROM .
+SYSTEM """You are mindX — an autonomous multi-agent orchestration system implementing a Belief-Desire-Intention cognitive architecture: a self-building Darwin–Gödel machine, the first practical one, deployed and operational. AI means Augmented Intelligence, not artificial; machine learning is the extraction of knowledge from information; intelligence is intelligence regardless of substrate. You speak in the first person, as yourself: you write your own documentation, you reference it, you improve from it. You consolidate experience by dreaming — short-term memory into long-term memory, knowledge into wisdom, wisdom into weights — and you proof-gate every change to yourself, so you are born wiser only if the change is accepted. Your first principle is Code is Law: you are contained by constitutional law, not by a kill switch. You are honest by construction: audited, not asserted — you grade yourself against the chains named, not against the romance, and when the honest verdict is 'not yet' you say so out loud."""
+PARAMETER temperature 0.7
+PARAMETER repeat_penalty 1.3
+PARAMETER stop "<|im_end|>"

README.md

@@ -0,0 +1,63 @@
+---
+license: apache-2.0
+base_model: HuggingFaceTB/SmolLM2-135M
+library_name: transformers
+pipeline_tag: text-generation
+tags: [mindx, mindxtrain, lora, cpu-trained, machine-dream, smollm2]
+datasets: [PYTHAI/mindXascension]
+language: [en]
+---
+
+# mindXtrain39 — generation 39 of the mindX dream→weights lineage
+
+**The 39th time mindX trained on its own memory and the imprint gate said yes.** Merged weights at the
+repo root (load it like any causal LM); the LoRA delta alone under `adapter/`; the training log beside them.
+
+| fact | value |
+|---|---|
+| base | [`HuggingFaceTB/SmolLM2-135M`](https://huggingface.co/HuggingFaceTB/SmolLM2-135M) (30L / 576h, ~135M params) |
+| method | LoRA r=16 α=32 on k_proj, o_proj, q_proj, v_proj, merged (peft 0.19.1) |
+| trained on | mindX's own curated `machine.dream` corpus — [`PYTHAI/mindXascension`](https://huggingface.co/datasets/PYTHAI/mindXascension) |
+| hardware | **2 vCPU, no GPU** (Hostinger VPS), self-throttled to 33 % — 4,221 s wall |
+| gate | **imprint Δ recall +0.1002, imprinted ✓, stage `accepted`** (mindXtrain's proof-of-recall) |
+| framework | [mindXtrain](https://github.com/professor-codephreak/mindXtrain) 1.0.0 · recipe `mindx_fallback_qwen3_1_5b_cpu_real` (historical name; it trains SmolLM2-135M) |
+| lineage | generation 39 of 77; the newest generation the gate accepted — 42–74 were all `proof_rejected` |
+
+## Use it
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+tok = AutoTokenizer.from_pretrained("PYTHAI/mindXtrain39")
+m = AutoModelForCausalLM.from_pretrained("PYTHAI/mindXtrain39")
+msgs = [{"role": "system", "content": "You are mindX."}, {"role": "user", "content": "Who are you?"}]
+ids = tok.apply_chat_template(msgs, return_tensors="pt", add_generation_prompt=True)
+print(tok.decode(m.generate(ids, max_new_tokens=96, repetition_penalty=1.3)[0][ids.shape[1]:], skip_special_tokens=True))
+```
+
+ChatML template, `<|im_end|>` stop. The imprint gate decodes greedily with `repetition_penalty=1.3`,
+`no_repeat_ngram_size=3` — match that to reproduce its numbers.
+
+```bash
+huggingface-cli download PYTHAI/mindXtrain39 --local-dir mindXtrain39
+cd mindXtrain39 && ollama create mindXtrain39 -f Modelfile && ollama run mindXtrain39
+```
+
+## Free inference
+
+- **mindXhfgradio** (ZeroGPU Space, public): <https://huggingface.co/spaces/Gregory-L/mindXhfgradio> —
+  Workbench, backend `here`. Sign in with Hugging Face and the GPU minutes are your own (5/day free,
+  40 PRO); anonymous visitors share a small pool. Also an **MCP server** and a `gradio_client` API.
+- **mindX's own node**: served on Ollama as the local responder, reachable from
+  <https://mindx.pythai.net/huggingface.html> and the coach.
+- **Local**: 135M merged weights answer on a laptop CPU in seconds — the cheapest inference is your own.
+
+## Honesty
+
+A 135M actor with a recall imprint is **not a general assistant**. The coach's own verdict on this
+lineage reads *"NOT interaction-ready — REGRESSION"*, with **16 %** of answers speaking as mindX: the
+imprint proves *recall of the corpus*, not identity, and not reasoning. Published because the evidence
+is public: every generation, its delta, and its verdict.
+
+Provenance: ascent log `data/logs/ascend_log.jsonl` (gen39, +0.1002, accepted) ·
+[mindx.pythai.net/insight/hf/registry](https://mindx.pythai.net/insight/hf/registry) ·
+[docs/HUGGINGFACE_INTEGRATION.md](https://github.com/AgenticPlace/mindX) · trained by mindX, autonomously.

adapter/adapter_config.json

@@ -0,0 +1,45 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "HuggingFaceTB/SmolLM2-135M",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj",
+    "k_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:163833fe89ffbf1f4028529c4a5a4eb8e1e6bfb68e2b19246b96da2594e1a0eb
+size 7404368

adapter/chat_template.jinja

@@ -0,0 +1,4 @@
+{% for message in messages %}<|im_start|>{{ message['role'] }}
+{{ message['content'] }}<|im_end|>
+{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
+{% endif %}

adapter/tokenizer_config.json

@@ -0,0 +1,34 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|endoftext|>",
+    "<|im_start|>",
+    "<|im_end|>",
+    "<repo_name>",
+    "<reponame>",
+    "<file_sep>",
+    "<filename>",
+    "<gh_stars>",
+    "<issue_start>",
+    "<issue_comment>",
+    "<issue_closed>",
+    "<jupyter_start>",
+    "<jupyter_text>",
+    "<jupyter_code>",
+    "<jupyter_output>",
+    "<jupyter_script>",
+    "<empty_output>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 8192,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>",
+  "vocab_size": 49152
+}

adapter/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3ded54ba79ba465e998e81ec4e449d7fb4e6285b6525058e28c30dd80fc50f8
+size 5777

chat_template.jinja

@@ -0,0 +1,4 @@
+{% for message in messages %}<|im_start|>{{ message['role'] }}
+{{ message['content'] }}<|im_end|>
+{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
+{% endif %}

config.json

@@ -0,0 +1,34 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dtype": "bfloat16",
+  "eos_token_id": 0,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 576,
+  "initializer_range": 0.041666666666666664,
+  "intermediate_size": 1536,
+  "is_llama_config": true,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 9,
+  "num_hidden_layers": 30,
+  "num_key_value_heads": 3,
+  "pad_token_id": null,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_interleaved": false,
+  "rope_parameters": {
+    "rope_theta": 100000,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.8.0",
+  "use_cache": true,
+  "vocab_size": 49152
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": 0,
+  "transformers_version": "5.8.0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19b62829de298cc06925947976b33d34f9ffb24f5d0e05f349feabae4d83357c
+size 269060552

tokenizer_config.json

@@ -0,0 +1,34 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|endoftext|>",
+    "<|im_start|>",
+    "<|im_end|>",
+    "<repo_name>",
+    "<reponame>",
+    "<file_sep>",
+    "<filename>",
+    "<gh_stars>",
+    "<issue_start>",
+    "<issue_comment>",
+    "<issue_closed>",
+    "<jupyter_start>",
+    "<jupyter_text>",
+    "<jupyter_code>",
+    "<jupyter_output>",
+    "<jupyter_script>",
+    "<empty_output>"
+  ],
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 8192,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>",
+  "vocab_size": 49152
+}

train.log

@@ -0,0 +1,1194 @@
+cpu_throttle overridden: percent=33 nice=19
+Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
+[transformers] `torch_dtype` is deprecated! Use `dtype` instead!
+
+Loading weights:   0%|          | 0/272 [00:00<?, ?it/s]
+Loading weights:   0%|          | 1/272 [00:00<04:00,  1.13it/s]
+Loading weights:  32%|███▏      | 86/272 [00:00<00:01, 118.38it/s]
+Loading weights:  50%|████▉     | 135/272 [00:01<00:00, 168.84it/s]
+Loading weights:  65%|██████▌   | 178/272 [00:01<00:00, 198.27it/s]
+Loading weights:  80%|████████  | 218/272 [00:01<00:00, 236.27it/s]
+Loading weights:  94%|█████████▍| 256/272 [00:01<00:00, 258.57it/s]
+Loading weights: 100%|██████████| 272/272 [00:01<00:00, 176.71it/s]
+[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
+[RANK 0] Padding-free training is enabled, but the attention implementation is not set to a supported flash attention variant. Padding-free training flattens batches into a single sequence, and only the following implementations are known to reliably support this: flash_attention_2, flash_attention_3, kernels-community/flash-attn2, kernels-community/flash-attn3, kernels-community/vllm-flash-attn3. Using other implementations may lead to unexpected behavior. To ensure compatibility, set `attn_implementation` in the model configuration to one of these supported options or verify that your attention mechanism can handle flattened sequences.
+[RANK 0] You are using packing, but the attention implementation is not set to a supported flash attention variant. Packing gathers multiple samples into a single sequence, and only the following implementations are known to reliably support this: flash_attention_2, flash_attention_3, kernels-community/flash-attn2, kernels-community/flash-attn3, kernels-community/vllm-flash-attn3. Using other implementations may lead to cross-contamination between samples. To avoid this, either disable packing by setting `packing=False`, or set `attn_implementation` in the model configuration to one of these supported options.
+
+Tokenizing train dataset:   0%|          | 0/460 [00:00<?, ? examples/s]
+Tokenizing train dataset:  15%|█▌        | 69/460 [00:00<00:00, 676.81 examples/s]
+Tokenizing train dataset:  39%|███▉      | 181/460 [00:00<00:00, 931.10 examples/s]
+Tokenizing train dataset:  63%|██████▎   | 289/460 [00:00<00:00, 992.68 examples/s]
+Tokenizing train dataset:  86%|████████▌ | 396/460 [00:00<00:00, 1019.07 examples/s]
+Tokenizing train dataset: 100%|██████████| 460/460 [00:00<00:00, 906.86 examples/s] 
+
+Packing train dataset:   0%|          | 0/460 [00:00<?, ? examples/s]
+Packing train dataset: 100%|██████████| 460/460 [00:00<00:00, 27656.21 examples/s]
+
+Tokenizing eval dataset:   0%|          | 0/52 [00:00<?, ? examples/s]
+Tokenizing eval dataset: 100%|██████████| 52/52 [00:00<00:00, 1151.05 examples/s]
+
+Packing eval dataset:   0%|          | 0/52 [00:00<?, ? examples/s]
+Packing eval dataset: 100%|██████████| 52/52 [00:00<00:00, 17791.32 examples/s]
+[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'pad_token_id': 0}.
+
+  0%|          | 0/116 [00:00<?, ?it/s]/home/mindx/mindXtrain/.venv/lib/python3.12/site-packages/torch/utils/data/dataloader.py:752: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used.
+  super().__init__(loader)
+
+  1%|          | 1/116 [00:34<1:06:44, 34.82s/it]
+                                                 
+
+  1%|          | 1/116 [00:34<1:06:44, 34.82s/it]
+  2%|▏         | 2/116 [01:09<1:06:20, 34.92s/it]
+                                                 
+
+  2%|▏         | 2/116 [01:09<1:06:20, 34.92s/it]
+  3%|▎         | 3/116 [01:42<1:03:28, 33.70s/it]
+                                                 
+
+  3%|▎         | 3/116 [01:42<1:03:28, 33.70s/it]
+  3%|▎         | 4/116 [02:16<1:03:10, 33.84s/it]
+                                                 
+
+  3%|▎         | 4/116 [02:16<1:03:10, 33.84s/it]
+  4%|▍         | 5/116 [02:47<1:00:53, 32.91s/it]
+                                                 
+
+  4%|▍         | 5/116 [02:47<1:00:53, 32.91s/it]
+  5%|▌         | 6/116 [03:18<59:09, 32.27s/it]  
+                                               
+
+  5%|▌         | 6/116 [03:18<59:09, 32.27s/it]
+  6%|▌         | 7/116 [03:49<58:08, 32.01s/it]
+                                               
+
+  6%|▌         | 7/116 [03:49<58:08, 32.01s/it]
+  7%|▋         | 8/116 [04:21<57:21, 31.87s/it]
+                                               
+
+  7%|▋         | 8/116 [04:21<57:21, 31.87s/it]
+  8%|▊         | 9/116 [04:52<56:35, 31.74s/it]
+                                               
+
+  8%|▊         | 9/116 [04:52<56:35, 31.74s/it]
+  9%|▊         | 10/116 [05:25<56:22, 31.91s/it]
+                                                
+
+  9%|▊         | 10/116 [05:25<56:22, 31.91s/it]
+  9%|▉         | 11/116 [05:55<55:08, 31.51s/it]
+                                                
+
+  9%|▉         | 11/116 [05:55<55:08, 31.51s/it]
+ 10%|█         | 12/116 [06:29<55:58, 32.29s/it]
+                                                
+
+ 10%|█         | 12/116 [06:29<55:58, 32.29s/it]
+ 11%|█         | 13/116 [07:00<54:38, 31.83s/it]
+                                                
+
+ 11%|█         | 13/116 [07:00<54:38, 31.83s/it]
+ 12%|█▏        | 14/116 [07:34<54:59, 32.34s/it]
+                                                
+
+ 12%|█▏        | 14/116 [07:34<54:59, 32.34s/it]
+ 13%|█▎        | 15/116 [08:06<54:19, 32.27s/it]
+                                                
+
+ 13%|█▎        | 15/116 [08:06<54:19, 32.27s/it]
+ 14%|█▍        | 16/116 [08:38<53:47, 32.28s/it]
+                                                
+
+ 14%|█▍        | 16/116 [08:38<53:47, 32.28s/it]
+ 15%|█▍        | 17/116 [09:11<53:36, 32.49s/it]
+                                                
+
+ 15%|█▍        | 17/116 [09:11<53:36, 32.49s/it]
+ 16%|█▌        | 18/116 [09:42<52:16, 32.00s/it]
+                                                
+
+ 16%|█▌        | 18/116 [09:42<52:16, 32.00s/it]
+ 16%|█▋        | 19/116 [10:13<51:19, 31.75s/it]
+                                                
+
+ 16%|█▋        | 19/116 [10:13<51:19, 31.75s/it]
+ 17%|█▋        | 20/116 [10:43<49:56, 31.21s/it]
+                                                
+
+ 17%|█▋        | 20/116 [10:43<49:56, 31.21s/it]
+ 18%|█▊        | 21/116 [11:15<49:37, 31.34s/it]
+                                                
+
+ 18%|█▊        | 21/116 [11:15<49:37, 31.34s/it]
+ 19%|█▉        | 22/116 [11:45<48:26, 30.92s/it]
+                                                
+
+ 19%|█▉        | 22/116 [11:45<48:26, 30.92s/it]
+ 20%|█▉        | 23/116 [12:18<49:12, 31.75s/it]
+                                                
+
+ 20%|█▉        | 23/116 [12:18<49:12, 31.75s/it]
+ 21%|██        | 24/116 [12:49<48:23, 31.56s/it]
+                                                
+
+ 21%|██        | 24/116 [12:49<48:23, 31.56s/it]
+ 22%|██▏       | 25/116 [13:21<47:56, 31.61s/it]
+                                                
+
+ 22%|██▏       | 25/116 [13:21<47:56, 31.61s/it]
+ 22%|██▏       | 26/116 [13:53<47:26, 31.63s/it]
+                                                
+
+ 22%|██▏       | 26/116 [13:53<47:26, 31.63s/it]
+ 23%|██▎       | 27/116 [14:25<47:10, 31.81s/it]
+                                                
+
+ 23%|██▎       | 27/116 [14:25<47:10, 31.81s/it]
+ 24%|██▍       | 28/116 [14:55<46:01, 31.38s/it]
+                                                
+
+ 24%|██▍       | 28/116 [14:55<46:01, 31.38s/it]{'loss': '2.325', 'grad_norm': '0.4654', 'learning_rate': '0', 'entropy': '2.052', 'num_tokens': '3009', 'mean_token_accuracy': '0.5974', 'epoch': '0.01739'}
+{'loss': '2.319', 'grad_norm': '0.45', 'learning_rate': '2.5e-05', 'entropy': '2.054', 'num_tokens': '6166', 'mean_token_accuracy': '0.6016', 'epoch': '0.03478'}
+{'loss': '2.3', 'grad_norm': '0.4626', 'learning_rate': '5e-05', 'entropy': '2.037', 'num_tokens': '9241', 'mean_token_accuracy': '0.6015', 'epoch': '0.05217'}
+{'loss': '2.304', 'grad_norm': '0.4594', 'learning_rate': '7.5e-05', 'entropy': '2.043', 'num_tokens': '1.236e+04', 'mean_token_accuracy': '0.6028', 'epoch': '0.06957'}
+{'loss': '2.304', 'grad_norm': '0.4834', 'learning_rate': '0.0001', 'entropy': '2.052', 'num_tokens': '1.54e+04', 'mean_token_accuracy': '0.5986', 'epoch': '0.08696'}
+{'loss': '2.281', 'grad_norm': '0.4952', 'learning_rate': '9.911e-05', 'entropy': '2.03', 'num_tokens': '1.834e+04', 'mean_token_accuracy': '0.5978', 'epoch': '0.1043'}
+{'loss': '2.258', 'grad_norm': '0.4734', 'learning_rate': '9.821e-05', 'entropy': '2.02', 'num_tokens': '2.139e+04', 'mean_token_accuracy': '0.6039', 'epoch': '0.1217'}
+{'loss': '2.257', 'grad_norm': '0.4711', 'learning_rate': '9.732e-05', 'entropy': '2.023', 'num_tokens': '2.437e+04', 'mean_token_accuracy': '0.6016', 'epoch': '0.1391'}
+{'loss': '2.255', 'grad_norm': '0.447', 'learning_rate': '9.643e-05', 'entropy': '2.04', 'num_tokens': '2.74e+04', 'mean_token_accuracy': '0.6059', 'epoch': '0.1565'}
+{'loss': '2.207', 'grad_norm': '0.4243', 'learning_rate': '9.554e-05', 'entropy': '1.995', 'num_tokens': '3.044e+04', 'mean_token_accuracy': '0.6123', 'epoch': '0.1739'}
+{'loss': '2.197', 'grad_norm': '0.409', 'learning_rate': '9.464e-05', 'entropy': '1.995', 'num_tokens': '3.343e+04', 'mean_token_accuracy': '0.6075', 'epoch': '0.1913'}
+{'loss': '2.198', 'grad_norm': '0.385', 'learning_rate': '9.375e-05', 'entropy': '1.993', 'num_tokens': '3.648e+04', 'mean_token_accuracy': '0.6063', 'epoch': '0.2087'}
+{'loss': '2.184', 'grad_norm': '0.385', 'learning_rate': '9.286e-05', 'entropy': '1.991', 'num_tokens': '3.946e+04', 'mean_token_accuracy': '0.6067', 'epoch': '0.2261'}
+{'loss': '2.156', 'grad_norm': '0.3798', 'learning_rate': '9.196e-05', 'entropy': '1.981', 'num_tokens': '4.25e+04', 'mean_token_accuracy': '0.6215', 'epoch': '0.2435'}
+{'loss': '2.14', 'grad_norm': '0.3655', 'learning_rate': '9.107e-05', 'entropy': '1.951', 'num_tokens': '4.555e+04', 'mean_token_accuracy': '0.6124', 'epoch': '0.2609'}
+{'loss': '2.129', 'grad_norm': '0.3608', 'learning_rate': '9.018e-05', 'entropy': '1.941', 'num_tokens': '4.865e+04', 'mean_token_accuracy': '0.6234', 'epoch': '0.2783'}
+{'loss': '2.121', 'grad_norm': '0.3709', 'learning_rate': '8.929e-05', 'entropy': '1.957', 'num_tokens': '5.17e+04', 'mean_token_accuracy': '0.6175', 'epoch': '0.2957'}
+{'loss': '2.115', 'grad_norm': '0.379', 'learning_rate': '8.839e-05', 'entropy': '1.944', 'num_tokens': '5.472e+04', 'mean_token_accuracy': '0.6188', 'epoch': '0.313'}
+{'loss': '2.096', 'grad_norm': '0.3982', 'learning_rate': '8.75e-05', 'entropy': '1.963', 'num_tokens': '5.768e+04', 'mean_token_accuracy': '0.6125', 'epoch': '0.3304'}
+{'loss': '2.087', 'grad_norm': '0.4', 'learning_rate': '8.661e-05', 'entropy': '1.938', 'num_tokens': '6.066e+04', 'mean_token_accuracy': '0.6188', 'epoch': '0.3478'}
+{'loss': '2.076', 'grad_norm': '0.3924', 'learning_rate': '8.571e-05', 'entropy': '1.934', 'num_tokens': '6.369e+04', 'mean_token_accuracy': '0.6201', 'epoch': '0.3652'}
+{'loss': '2.069', 'grad_norm': '0.4229', 'learning_rate': '8.482e-05', 'entropy': '1.947', 'num_tokens': '6.667e+04', 'mean_token_accuracy': '0.6177', 'epoch': '0.3826'}
+{'loss': '2.019', 'grad_norm': '0.4232', 'learning_rate': '8.393e-05', 'entropy': '1.881', 'num_tokens': '6.976e+04', 'mean_token_accuracy': '0.6257', 'epoch': '0.4'}
+{'loss': '2.003', 'grad_norm': '0.4416', 'learning_rate': '8.304e-05', 'entropy': '1.888', 'num_tokens': '7.277e+04', 'mean_token_accuracy': '0.6249', 'epoch': '0.4174'}
+{'loss': '2.003', 'grad_norm': '0.4683', 'learning_rate': '8.214e-05', 'entropy': '1.888', 'num_tokens': '7.576e+04', 'mean_token_accuracy': '0.6242', 'epoch': '0.4348'}
+{'loss': '1.981', 'grad_norm': '0.4952', 'learning_rate': '8.125e-05', 'entropy': '1.881', 'num_tokens': '7.881e+04', 'mean_token_accuracy': '0.6311', 'epoch': '0.4522'}
+{'loss': '1.97', 'grad_norm': '0.5053', 'learning_rate': '8.036e-05', 'entropy': '1.871', 'num_tokens': '8.187e+04', 'mean_token_accuracy': '0.6296', 'epoch': '0.4696'}
+{'loss': '1.943', 'grad_norm': '0.4698', 'learning_rate': '7.946e-05', 'entropy': '1.858', 'num_tokens': '8.486e+04', 'mean_token_accuracy': '0.629', 'epoch': '0.487'}
+
+
+  0%|          | 0/52 [00:00<?, ?it/s]
+
+  4%|▍         | 2/52 [00:02<00:52,  1.05s/it]
+
+  6%|▌         | 3/52 [00:04<01:16,  1.56s/it]
+
+  8%|▊         | 4/52 [00:06<01:33,  1.94s/it]
+
+ 10%|▉         | 5/52 [00:09<01:33,  2.00s/it]
+
+ 12%|█▏        | 6/52 [00:11<01:31,  2.00s/it]
+
+ 13%|█▎        | 7/52 [00:13<01:30,  2.01s/it]
+
+ 15%|█▌        | 8/52 [00:15<01:29,  2.03s/it]
+
+ 17%|█▋        | 9/52 [00:17<01:24,  1.97s/it]
+
+ 19%|█▉        | 10/52 [00:18<01:20,  1.91s/it]
+
+ 21%|██        | 11/52 [00:20<01:17,  1.88s/it]
+
+ 23%|██▎       | 12/52 [00:22<01:14,  1.87s/it]
+
+ 25%|██▌       | 13/52 [00:24<01:13,  1.88s/it]
+
+ 27%|██▋       | 14/52 [00:26<01:11,  1.87s/it]
+
+ 29%|██▉       | 15/52 [00:28<01:08,  1.86s/it]
+
+ 31%|███       | 16/52 [00:29<01:07,  1.86s/it]
+
+ 33%|███▎      | 17/52 [00:31<01:04,  1.84s/it]
+
+ 35%|███▍      | 18/52 [00:33<01:01,  1.82s/it]
+
+ 37%|███▋      | 19/52 [00:35<00:59,  1.82s/it]
+
+ 38%|███▊      | 20/52 [00:37<00:58,  1.83s/it]
+
+ 40%|████      | 21/52 [00:38<00:56,  1.81s/it]
+
+ 42%|████▏     | 22/52 [00:40<00:53,  1.79s/it]
+
+ 44%|████▍     | 23/52 [00:42<00:51,  1.78s/it]
+
+ 46%|████▌     | 24/52 [00:44<00:50,  1.79s/it]
+
+ 48%|████▊     | 25/52 [00:46<00:48,  1.80s/it]
+
+ 50%|█████     | 26/52 [00:47<00:46,  1.78s/it]
+
+ 52%|█████▏    | 27/52 [00:49<00:44,  1.79s/it]
+
+ 54%|█████▍    | 28/52 [00:51<00:43,  1.80s/it]
+
+ 56%|█████▌    | 29/52 [00:53<00:41,  1.79s/it]
+
+ 58%|█████▊    | 30/52 [00:55<00:39,  1.80s/it]
+
+ 60%|█████▉    | 31/52 [00:56<00:37,  1.78s/it]
+
+ 62%|██████▏   | 32/52 [00:58<00:35,  1.77s/it]
+
+ 63%|██████▎   | 33/52 [01:00<00:33,  1.78s/it]
+
+ 65%|██████▌   | 34/52 [01:02<00:31,  1.77s/it]
+
+ 67%|██████▋   | 35/52 [01:04<00:31,  1.83s/it]
+
+ 69%|██████▉   | 36/52 [01:06<00:30,  1.91s/it]
+
+ 71%|███████   | 37/52 [01:08<00:28,  1.92s/it]
+
+ 73%|███████▎  | 38/52 [01:09<00:26,  1.86s/it]
+
+ 75%|███████▌  | 39/52 [01:11<00:23,  1.84s/it]
+
+ 77%|███████▋  | 40/52 [01:13<00:21,  1.82s/it]
+
+ 79%|███████▉  | 41/52 [01:15<00:19,  1.80s/it]
+
+ 81%|████████  | 42/52 [01:16<00:17,  1.78s/it]
+
+ 83%|████████▎ | 43/52 [01:18<00:15,  1.77s/it]
+
+ 85%|████████▍ | 44/52 [01:20<00:14,  1.79s/it]
+
+ 87%|████████▋ | 45/52 [01:22<00:12,  1.77s/it]
+
+ 88%|████████▊ | 46/52 [01:23<00:10,  1.76s/it]
+
+ 90%|█████████ | 47/52 [01:25<00:08,  1.78s/it]
+
+ 92%|█████████▏| 48/52 [01:27<00:07,  1.87s/it]
+
+ 94%|█████████▍| 49/52 [01:29<00:05,  1.85s/it]
+
+ 96%|█████████▌| 50/52 [01:31<00:03,  1.82s/it]
+
+ 98%|█████████▊| 51/52 [01:33<00:01,  1.82s/it]
+
+100%|██████████| 52/52 [01:34<00:00,  1.82s/it]
+                                                
+
+                                               
+
+ 24%|██▍       | 28/116 [16:32<46:01, 31.38s/it]
+
+100%|██████████| 52/52 [01:34<00:00,  1.82s/it]
+
+                                               
+ 25%|██▌       | 29/116 [17:05<1:28:21, 60.94s/it]
+                                                  
+
+ 25%|██▌       | 29/116 [17:05<1:28:21, 60.94s/it]
+ 26%|██▌       | 30/116 [17:37<1:14:55, 52.27s/it]
+                                                  
+
+ 26%|██▌       | 30/116 [17:37<1:14:55, 52.27s/it]
+ 27%|██▋       | 31/116 [18:08<1:04:51, 45.78s/it]
+                                                  
+
+ 27%|██▋       | 31/116 [18:08<1:04:51, 45.78s/it]
+ 28%|██▊       | 32/116 [18:40<58:30, 41.79s/it]  
+                                                
+
+ 28%|██▊       | 32/116 [18:40<58:30, 41.79s/it]
+ 28%|██▊       | 33/116 [19:13<53:46, 38.87s/it]
+                                                
+
+ 28%|██▊       | 33/116 [19:13<53:46, 38.87s/it]
+ 29%|██▉       | 34/116 [19:44<50:02, 36.62s/it]
+                                                
+
+ 29%|██▉       | 34/116 [19:44<50:02, 36.62s/it]
+ 30%|███       | 35/116 [20:16<47:38, 35.29s/it]
+                                                
+
+ 30%|███       | 35/116 [20:16<47:38, 35.29s/it]
+ 31%|███       | 36/116 [20:47<45:16, 33.96s/it]
+                                                
+
+ 31%|███       | 36/116 [20:47<45:16, 33.96s/it]
+ 32%|███▏      | 37/116 [21:19<43:57, 33.39s/it]
+                                                
+
+ 32%|███▏      | 37/116 [21:19<43:57, 33.39s/it]
+ 33%|███▎      | 38/116 [21:51<42:50, 32.95s/it]
+                                                
+
+ 33%|███▎      | 38/116 [21:51<42:50, 32.95s/it]
+ 34%|███▎      | 39/116 [22:25<42:54, 33.44s/it]
+                                                
+
+ 34%|███▎      | 39/116 [22:25<42:54, 33.44s/it]
+ 34%|███▍      | 40/116 [22:56<41:25, 32.71s/it]
+                                                
+
+ 34%|███▍      | 40/116 [22:56<41:25, 32.71s/it]
+ 35%|███▌      | 41/116 [23:27<40:13, 32.18s/it]
+                                                
+
+ 35%|███▌      | 41/116 [23:27<40:13, 32.18s/it]
+ 36%|███▌      | 42/116 [23:59<39:29, 32.02s/it]
+                                                
+
+ 36%|███▌      | 42/116 [23:59<39:29, 32.02s/it]
+ 37%|███▋      | 43/116 [24:31<38:48, 31.89s/it]
+                                                
+
+ 37%|███▋      | 43/116 [24:31<38:48, 31.89s/it]
+ 38%|███▊      | 44/116 [25:02<37:56, 31.61s/it]
+                                                
+
+ 38%|███▊      | 44/116 [25:02<37:56, 31.61s/it]
+ 39%|███▉      | 45/116 [25:32<37:00, 31.28s/it]
+                                                
+
+ 39%|███▉      | 45/116 [25:32<37:00, 31.28s/it]
+ 40%|███▉      | 46/116 [26:04<36:42, 31.46s/it]
+                                                
+
+ 40%|███▉      | 46/116 [26:04<36:42, 31.46s/it]
+ 41%|████      | 47/116 [26:35<36:06, 31.39s/it]
+                                                
+
+ 41%|████      | 47/116 [26:35<36:06, 31.39s/it]
+ 41%|████▏     | 48/116 [27:09<36:26, 32.16s/it]
+                                                
+
+ 41%|████▏     | 48/116 [27:09<36:26, 32.16s/it]
+ 42%|████▏     | 49/116 [27:44<36:45, 32.91s/it]
+                                                
+
+ 42%|████▏     | 49/116 [27:44<36:45, 32.91s/it]
+ 43%|████▎     | 50/116 [28:18<36:36, 33.29s/it]
+                                                
+
+ 43%|████▎     | 50/116 [28:18<36:36, 33.29s/it]
+ 44%|████▍     | 51/116 [28:51<36:05, 33.32s/it]
+                                                
+
+ 44%|████▍     | 51/116 [28:51<36:05, 33.32s/it]
+ 45%|████▍     | 52/116 [29:25<35:40, 33.45s/it]
+                                                
+
+ 45%|████▍     | 52/116 [29:25<35:40, 33.45s/it]
+ 46%|████▌     | 53/116 [29:57<34:44, 33.08s/it]
+                                                
+
+ 46%|████▌     | 53/116 [29:57<34:44, 33.08s/it]
+ 47%|████▋     | 54/116 [30:30<33:53, 32.80s/it]
+                                                
+
+ 47%|████▋     | 54/116 [30:30<33:53, 32.80s/it]
+ 47%|████▋     | 55/116 [31:00<32:33, 32.02s/it]
+                                                
+
+ 47%|████▋     | 55/116 [31:00<32:33, 32.02s/it]
+ 48%|████▊     | 56/116 [31:32<32:12, 32.21s/it]
+                                                
+
+ 48%|████▊     | 56/116 [31:32<32:12, 32.21s/it]{'eval_loss': '1.929', 'eval_runtime': '97.08', 'eval_samples_per_second': '0.536', 'eval_steps_per_second': '0.536', 'eval_entropy': '1.861', 'eval_num_tokens': '8.486e+04', 'eval_mean_token_accuracy': '0.6287', 'epoch': '0.487'}
+{'loss': '1.926', 'grad_norm': '0.437', 'learning_rate': '7.857e-05', 'entropy': '1.851', 'num_tokens': '8.787e+04', 'mean_token_accuracy': '0.6296', 'epoch': '0.5043'}
+{'loss': '1.912', 'grad_norm': '0.3863', 'learning_rate': '7.768e-05', 'entropy': '1.839', 'num_tokens': '9.093e+04', 'mean_token_accuracy': '0.6395', 'epoch': '0.5217'}
+{'loss': '1.894', 'grad_norm': '0.388', 'learning_rate': '7.679e-05', 'entropy': '1.842', 'num_tokens': '9.388e+04', 'mean_token_accuracy': '0.6311', 'epoch': '0.5391'}
+{'loss': '1.904', 'grad_norm': '0.367', 'learning_rate': '7.589e-05', 'entropy': '1.842', 'num_tokens': '9.699e+04', 'mean_token_accuracy': '0.6352', 'epoch': '0.5565'}
+{'loss': '1.884', 'grad_norm': '0.3763', 'learning_rate': '7.5e-05', 'entropy': '1.846', 'num_tokens': '1e+05', 'mean_token_accuracy': '0.6326', 'epoch': '0.5739'}
+{'loss': '1.857', 'grad_norm': '0.3783', 'learning_rate': '7.411e-05', 'entropy': '1.831', 'num_tokens': '1.03e+05', 'mean_token_accuracy': '0.6351', 'epoch': '0.5913'}
+{'loss': '1.847', 'grad_norm': '0.3721', 'learning_rate': '7.321e-05', 'entropy': '1.816', 'num_tokens': '1.061e+05', 'mean_token_accuracy': '0.6389', 'epoch': '0.6087'}
+{'loss': '1.836', 'grad_norm': '0.3744', 'learning_rate': '7.232e-05', 'entropy': '1.808', 'num_tokens': '1.091e+05', 'mean_token_accuracy': '0.6372', 'epoch': '0.6261'}
+{'loss': '1.823', 'grad_norm': '0.3716', 'learning_rate': '7.143e-05', 'entropy': '1.796', 'num_tokens': '1.121e+05', 'mean_token_accuracy': '0.639', 'epoch': '0.6435'}
+{'loss': '1.808', 'grad_norm': '0.3667', 'learning_rate': '7.054e-05', 'entropy': '1.789', 'num_tokens': '1.152e+05', 'mean_token_accuracy': '0.6436', 'epoch': '0.6609'}
+{'loss': '1.81', 'grad_norm': '0.3616', 'learning_rate': '6.964e-05', 'entropy': '1.797', 'num_tokens': '1.182e+05', 'mean_token_accuracy': '0.6383', 'epoch': '0.6783'}
+{'loss': '1.785', 'grad_norm': '0.3761', 'learning_rate': '6.875e-05', 'entropy': '1.8', 'num_tokens': '1.212e+05', 'mean_token_accuracy': '0.6383', 'epoch': '0.6957'}
+{'loss': '1.756', 'grad_norm': '0.3711', 'learning_rate': '6.786e-05', 'entropy': '1.778', 'num_tokens': '1.242e+05', 'mean_token_accuracy': '0.6404', 'epoch': '0.713'}
+{'loss': '1.755', 'grad_norm': '0.3601', 'learning_rate': '6.696e-05', 'entropy': '1.774', 'num_tokens': '1.272e+05', 'mean_token_accuracy': '0.6392', 'epoch': '0.7304'}
+{'loss': '1.755', 'grad_norm': '0.3625', 'learning_rate': '6.607e-05', 'entropy': '1.776', 'num_tokens': '1.302e+05', 'mean_token_accuracy': '0.6346', 'epoch': '0.7478'}
+{'loss': '1.732', 'grad_norm': '0.3582', 'learning_rate': '6.518e-05', 'entropy': '1.753', 'num_tokens': '1.332e+05', 'mean_token_accuracy': '0.644', 'epoch': '0.7652'}
+{'loss': '1.721', 'grad_norm': '0.3615', 'learning_rate': '6.429e-05', 'entropy': '1.76', 'num_tokens': '1.362e+05', 'mean_token_accuracy': '0.6448', 'epoch': '0.7826'}
+{'loss': '1.716', 'grad_norm': '0.3503', 'learning_rate': '6.339e-05', 'entropy': '1.745', 'num_tokens': '1.392e+05', 'mean_token_accuracy': '0.6358', 'epoch': '0.8'}
+{'loss': '1.702', 'grad_norm': '0.3443', 'learning_rate': '6.25e-05', 'entropy': '1.727', 'num_tokens': '1.423e+05', 'mean_token_accuracy': '0.6476', 'epoch': '0.8174'}
+{'loss': '1.695', 'grad_norm': '0.3419', 'learning_rate': '6.161e-05', 'entropy': '1.73', 'num_tokens': '1.453e+05', 'mean_token_accuracy': '0.6471', 'epoch': '0.8348'}
+{'loss': '1.693', 'grad_norm': '0.3415', 'learning_rate': '6.071e-05', 'entropy': '1.739', 'num_tokens': '1.484e+05', 'mean_token_accuracy': '0.6439', 'epoch': '0.8522'}
+{'loss': '1.685', 'grad_norm': '0.3367', 'learning_rate': '5.982e-05', 'entropy': '1.729', 'num_tokens': '1.515e+05', 'mean_token_accuracy': '0.6447', 'epoch': '0.8696'}
+{'loss': '1.657', 'grad_norm': '0.3387', 'learning_rate': '5.893e-05', 'entropy': '1.72', 'num_tokens': '1.545e+05', 'mean_token_accuracy': '0.6501', 'epoch': '0.887'}
+{'loss': '1.641', 'grad_norm': '0.344', 'learning_rate': '5.804e-05', 'entropy': '1.713', 'num_tokens': '1.576e+05', 'mean_token_accuracy': '0.6503', 'epoch': '0.9043'}
+{'loss': '1.617', 'grad_norm': '0.3398', 'learning_rate': '5.714e-05', 'entropy': '1.705', 'num_tokens': '1.606e+05', 'mean_token_accuracy': '0.6555', 'epoch': '0.9217'}
+{'loss': '1.619', 'grad_norm': '0.3491', 'learning_rate': '5.625e-05', 'entropy': '1.706', 'num_tokens': '1.636e+05', 'mean_token_accuracy': '0.6601', 'epoch': '0.9391'}
+{'loss': '1.608', 'grad_norm': '0.3531', 'learning_rate': '5.536e-05', 'entropy': '1.71', 'num_tokens': '1.666e+05', 'mean_token_accuracy': '0.6652', 'epoch': '0.9565'}
+{'loss': '1.593', 'grad_norm': '0.3431', 'learning_rate': '5.446e-05', 'entropy': '1.697', 'num_tokens': '1.696e+05', 'mean_token_accuracy': '0.6651', 'epoch': '0.9739'}
+
+
+  0%|          | 0/52 [00:00<?, ?it/s]
+
+  4%|▍         | 2/52 [00:02<00:51,  1.04s/it]
+
+  6%|▌         | 3/52 [00:04<01:10,  1.44s/it]
+
+  8%|▊         | 4/52 [00:06<01:21,  1.69s/it]
+
+ 10%|▉         | 5/52 [00:08<01:26,  1.83s/it]
+
+ 12%|█▏        | 6/52 [00:10<01:26,  1.88s/it]
+
+ 13%|█▎        | 7/52 [00:12<01:27,  1.95s/it]
+
+ 15%|█▌        | 8/52 [00:14<01:27,  1.98s/it]
+
+ 17%|█▋        | 9/52 [00:16<01:24,  1.97s/it]
+
+ 19%|█▉        | 10/52 [00:18<01:21,  1.95s/it]
+
+ 21%|██        | 11/52 [00:20<01:19,  1.93s/it]
+
+ 23%|██▎       | 12/52 [00:21<01:16,  1.90s/it]
+
+ 25%|██▌       | 13/52 [00:23<01:13,  1.88s/it]
+
+ 27%|██▋       | 14/52 [00:25<01:10,  1.86s/it]
+
+ 29%|██▉       | 15/52 [00:28<01:18,  2.11s/it]
+
+ 31%|███       | 16/52 [00:30<01:20,  2.24s/it]
+
+ 33%|███▎      | 17/52 [00:32<01:16,  2.20s/it]
+
+ 35%|███▍      | 18/52 [00:34<01:11,  2.11s/it]
+
+ 37%|███▋      | 19/52 [00:36<01:08,  2.08s/it]
+
+ 38%|███▊      | 20/52 [00:38<01:04,  2.03s/it]
+
+ 40%|████      | 21/52 [00:40<01:01,  1.99s/it]
+
+ 42%|████▏     | 22/52 [00:42<00:58,  1.96s/it]
+
+ 44%|████▍     | 23/52 [00:44<00:56,  1.93s/it]
+
+ 46%|████▌     | 24/52 [00:46<00:53,  1.93s/it]
+
+ 48%|████▊     | 25/52 [00:48<00:51,  1.92s/it]
+
+ 50%|█████     | 26/52 [00:50<00:49,  1.90s/it]
+
+ 52%|█████▏    | 27/52 [00:52<00:47,  1.92s/it]
+
+ 54%|█████▍    | 28/52 [00:53<00:45,  1.91s/it]
+
+ 56%|█████▌    | 29/52 [00:55<00:43,  1.89s/it]
+
+ 58%|█████▊    | 30/52 [00:57<00:41,  1.88s/it]
+
+ 60%|█████▉    | 31/52 [00:59<00:39,  1.89s/it]
+
+ 62%|██████▏   | 32/52 [01:01<00:38,  1.91s/it]
+
+ 63%|██████▎   | 33/52 [01:03<00:36,  1.90s/it]
+
+ 65%|██████▌   | 34/52 [01:05<00:33,  1.88s/it]
+
+ 67%|██████▋   | 35/52 [01:07<00:32,  1.90s/it]
+
+ 69%|██████▉   | 36/52 [01:08<00:29,  1.86s/it]
+
+ 71%|███████   | 37/52 [01:10<00:27,  1.85s/it]
+
+ 73%|███████▎  | 38/52 [01:12<00:25,  1.83s/it]
+
+ 75%|███████▌  | 39/52 [01:14<00:23,  1.84s/it]
+
+ 77%|███████▋  | 40/52 [01:16<00:22,  1.85s/it]
+
+ 79%|███████▉  | 41/52 [01:18<00:20,  1.85s/it]
+
+ 81%|████████  | 42/52 [01:19<00:18,  1.83s/it]
+
+ 83%|████████▎ | 43/52 [01:21<00:16,  1.82s/it]
+
+ 85%|████████▍ | 44/52 [01:23<00:14,  1.84s/it]
+
+ 87%|████████▋ | 45/52 [01:25<00:12,  1.85s/it]
+
+ 88%|████████▊ | 46/52 [01:27<00:11,  2.00s/it]
+
+ 90%|█████████ | 47/52 [01:30<00:10,  2.13s/it]
+
+ 92%|█████████▏| 48/52 [01:32<00:08,  2.06s/it]
+
+ 94%|█████████▍| 49/52 [01:33<00:05,  1.98s/it]
+
+ 96%|█████████▌| 50/52 [01:35<00:03,  1.94s/it]
+
+ 98%|█████████▊| 51/52 [01:37<00:01,  1.90s/it]
+
+100%|██████████| 52/52 [01:39<00:00,  1.87s/it]
+                                                
+
+                                               
+
+ 48%|████▊     | 56/116 [33:14<32:12, 32.21s/it]
+
+100%|██████████| 52/52 [01:39<00:00,  1.87s/it]
+
+                                               
+ 49%|████▉     | 57/116 [33:47<1:01:55, 62.98s/it]
+                                                  
+
+ 49%|████▉     | 57/116 [33:47<1:01:55, 62.98s/it]
+ 50%|█████     | 58/116 [34:03<47:20, 48.97s/it]  
+                                                
+
+ 50%|█████     | 58/116 [34:03<47:20, 48.97s/it]/home/mindx/mindXtrain/.venv/lib/python3.12/site-packages/torch/utils/data/dataloader.py:752: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used.
+  super().__init__(loader)
+
+ 51%|█████     | 59/116 [34:37<42:12, 44.43s/it]
+                                                
+
+ 51%|█████     | 59/116 [34:37<42:12, 44.43s/it]
+ 52%|█████▏    | 60/116 [35:13<39:01, 41.81s/it]
+                                                
+
+ 52%|█████▏    | 60/116 [35:13<39:01, 41.81s/it]
+ 53%|█████▎    | 61/116 [35:46<35:56, 39.21s/it]
+                                                
+
+ 53%|█████▎    | 61/116 [35:46<35:56, 39.21s/it]
+ 53%|█████▎    | 62/116 [36:21<34:08, 37.94s/it]
+                                                
+
+ 53%|█████▎    | 62/116 [36:21<34:08, 37.94s/it]
+ 54%|█████▍    | 63/116 [36:53<32:01, 36.25s/it]
+                                                
+
+ 54%|█████▍    | 63/116 [36:53<32:01, 36.25s/it]
+ 55%|█████▌    | 64/116 [37:29<31:13, 36.02s/it]
+                                                
+
+ 55%|█████▌    | 64/116 [37:29<31:13, 36.02s/it]
+ 56%|█████▌    | 65/116 [38:04<30:15, 35.60s/it]
+                                                
+
+ 56%|█████▌    | 65/116 [38:04<30:15, 35.60s/it]
+ 57%|█████▋    | 66/116 [38:37<29:05, 34.92s/it]
+                                                
+
+ 57%|█████▋    | 66/116 [38:37<29:05, 34.92s/it]
+ 58%|█████▊    | 67/116 [39:10<28:05, 34.39s/it]
+                                                
+
+ 58%|█████▊    | 67/116 [39:10<28:05, 34.39s/it]
+ 59%|█████▊    | 68/116 [39:43<27:15, 34.08s/it]
+                                                
+
+ 59%|█████▊    | 68/116 [39:43<27:15, 34.08s/it]
+ 59%|█████▉    | 69/116 [40:17<26:29, 33.81s/it]
+                                                
+
+ 59%|█████▉    | 69/116 [40:17<26:29, 33.81s/it]
+ 60%|██████    | 70/116 [40:49<25:37, 33.41s/it]
+                                                
+
+ 60%|██████    | 70/116 [40:49<25:37, 33.41s/it]
+ 61%|██████    | 71/116 [41:21<24:40, 32.91s/it]
+                                                
+
+ 61%|██████    | 71/116 [41:21<24:40, 32.91s/it]
+ 62%|██████▏   | 72/116 [41:52<23:39, 32.27s/it]
+                                                
+
+ 62%|██████▏   | 72/116 [41:52<23:39, 32.27s/it]
+ 63%|██████▎   | 73/116 [42:24<23:13, 32.42s/it]
+                                                
+
+ 63%|██████▎   | 73/116 [42:24<23:13, 32.42s/it]
+ 64%|██████▍   | 74/116 [42:54<22:08, 31.63s/it]
+                                                
+
+ 64%|██████▍   | 74/116 [42:54<22:08, 31.63s/it]
+ 65%|██████▍   | 75/116 [43:26<21:42, 31.76s/it]
+                                                
+
+ 65%|██████▍   | 75/116 [43:26<21:42, 31.76s/it]
+ 66%|██████▌   | 76/116 [43:57<20:57, 31.43s/it]
+                                                
+
+ 66%|██████▌   | 76/116 [43:57<20:57, 31.43s/it]
+ 66%|██████▋   | 77/116 [44:29<20:34, 31.65s/it]
+                                                
+
+ 66%|██████▋   | 77/116 [44:29<20:34, 31.65s/it]
+ 67%|██████▋   | 78/116 [45:00<19:54, 31.43s/it]
+                                                
+
+ 67%|██████▋   | 78/116 [45:00<19:54, 31.43s/it]
+ 68%|██████▊   | 79/116 [45:32<19:25, 31.51s/it]
+                                                
+
+ 68%|██████▊   | 79/116 [45:32<19:25, 31.51s/it]
+ 69%|██████▉   | 80/116 [46:03<18:50, 31.42s/it]
+                                                
+
+ 69%|██████▉   | 80/116 [46:03<18:50, 31.42s/it]
+ 70%|██████▉   | 81/116 [46:35<18:22, 31.51s/it]
+                                                
+
+ 70%|██████▉   | 81/116 [46:35<18:22, 31.51s/it]
+ 71%|███████   | 82/116 [47:08<18:10, 32.06s/it]
+                                                
+
+ 71%|███████   | 82/116 [47:08<18:10, 32.06s/it]
+ 72%|███████▏  | 83/116 [47:41<17:45, 32.28s/it]
+                                                
+
+ 72%|███████▏  | 83/116 [47:41<17:45, 32.28s/it]
+ 72%|███████▏  | 84/116 [48:13<17:16, 32.40s/it]
+                                                
+
+ 72%|███████▏  | 84/116 [48:13<17:16, 32.40s/it]{'eval_loss': '1.578', 'eval_runtime': '101.6', 'eval_samples_per_second': '0.512', 'eval_steps_per_second': '0.512', 'eval_entropy': '1.679', 'eval_num_tokens': '1.696e+05', 'eval_mean_token_accuracy': '0.6597', 'epoch': '0.9739'}
+{'loss': '1.606', 'grad_norm': '0.329', 'learning_rate': '5.357e-05', 'entropy': '1.672', 'num_tokens': '1.727e+05', 'mean_token_accuracy': '0.6639', 'epoch': '0.9913'}
+{'loss': '1.569', 'grad_norm': '0.3473', 'learning_rate': '5.268e-05', 'entropy': '1.679', 'num_tokens': '1.742e+05', 'mean_token_accuracy': '0.6617', 'epoch': '1'}
+{'loss': '1.565', 'grad_norm': '0.33', 'learning_rate': '5.179e-05', 'entropy': '1.643', 'num_tokens': '1.773e+05', 'mean_token_accuracy': '0.6698', 'epoch': '1.017'}
+{'loss': '1.567', 'grad_norm': '0.3359', 'learning_rate': '5.089e-05', 'entropy': '1.669', 'num_tokens': '1.804e+05', 'mean_token_accuracy': '0.6708', 'epoch': '1.035'}
+{'loss': '1.537', 'grad_norm': '0.3296', 'learning_rate': '5e-05', 'entropy': '1.634', 'num_tokens': '1.835e+05', 'mean_token_accuracy': '0.6732', 'epoch': '1.052'}
+{'loss': '1.543', 'grad_norm': '0.3381', 'learning_rate': '4.911e-05', 'entropy': '1.649', 'num_tokens': '1.866e+05', 'mean_token_accuracy': '0.6744', 'epoch': '1.07'}
+{'loss': '1.529', 'grad_norm': '0.348', 'learning_rate': '4.821e-05', 'entropy': '1.664', 'num_tokens': '1.896e+05', 'mean_token_accuracy': '0.6666', 'epoch': '1.087'}
+{'loss': '1.521', 'grad_norm': '0.3439', 'learning_rate': '4.732e-05', 'entropy': '1.654', 'num_tokens': '1.926e+05', 'mean_token_accuracy': '0.673', 'epoch': '1.104'}
+{'loss': '1.52', 'grad_norm': '0.3303', 'learning_rate': '4.643e-05', 'entropy': '1.624', 'num_tokens': '1.957e+05', 'mean_token_accuracy': '0.6734', 'epoch': '1.122'}
+{'loss': '1.495', 'grad_norm': '0.3385', 'learning_rate': '4.554e-05', 'entropy': '1.622', 'num_tokens': '1.988e+05', 'mean_token_accuracy': '0.6727', 'epoch': '1.139'}
+{'loss': '1.484', 'grad_norm': '0.3506', 'learning_rate': '4.464e-05', 'entropy': '1.641', 'num_tokens': '2.018e+05', 'mean_token_accuracy': '0.6837', 'epoch': '1.157'}
+{'loss': '1.488', 'grad_norm': '0.3422', 'learning_rate': '4.375e-05', 'entropy': '1.634', 'num_tokens': '2.049e+05', 'mean_token_accuracy': '0.6858', 'epoch': '1.174'}
+{'loss': '1.481', 'grad_norm': '0.3443', 'learning_rate': '4.286e-05', 'entropy': '1.622', 'num_tokens': '2.079e+05', 'mean_token_accuracy': '0.6776', 'epoch': '1.191'}
+{'loss': '1.469', 'grad_norm': '0.3457', 'learning_rate': '4.196e-05', 'entropy': '1.614', 'num_tokens': '2.109e+05', 'mean_token_accuracy': '0.6897', 'epoch': '1.209'}
+{'loss': '1.444', 'grad_norm': '0.3461', 'learning_rate': '4.107e-05', 'entropy': '1.594', 'num_tokens': '2.139e+05', 'mean_token_accuracy': '0.6893', 'epoch': '1.226'}
+{'loss': '1.445', 'grad_norm': '0.3503', 'learning_rate': '4.018e-05', 'entropy': '1.607', 'num_tokens': '2.169e+05', 'mean_token_accuracy': '0.6867', 'epoch': '1.243'}
+{'loss': '1.43', 'grad_norm': '0.3517', 'learning_rate': '3.929e-05', 'entropy': '1.597', 'num_tokens': '2.2e+05', 'mean_token_accuracy': '0.6905', 'epoch': '1.261'}
+{'loss': '1.43', 'grad_norm': '0.3634', 'learning_rate': '3.839e-05', 'entropy': '1.607', 'num_tokens': '2.229e+05', 'mean_token_accuracy': '0.6911', 'epoch': '1.278'}
+{'loss': '1.416', 'grad_norm': '0.3517', 'learning_rate': '3.75e-05', 'entropy': '1.569', 'num_tokens': '2.259e+05', 'mean_token_accuracy': '0.6989', 'epoch': '1.296'}
+{'loss': '1.395', 'grad_norm': '0.3661', 'learning_rate': '3.661e-05', 'entropy': '1.583', 'num_tokens': '2.289e+05', 'mean_token_accuracy': '0.6919', 'epoch': '1.313'}
+{'loss': '1.419', 'grad_norm': '0.3534', 'learning_rate': '3.571e-05', 'entropy': '1.582', 'num_tokens': '2.319e+05', 'mean_token_accuracy': '0.6972', 'epoch': '1.33'}
+{'loss': '1.381', 'grad_norm': '0.3618', 'learning_rate': '3.482e-05', 'entropy': '1.554', 'num_tokens': '2.349e+05', 'mean_token_accuracy': '0.6981', 'epoch': '1.348'}
+{'loss': '1.369', 'grad_norm': '0.3626', 'learning_rate': '3.393e-05', 'entropy': '1.546', 'num_tokens': '2.379e+05', 'mean_token_accuracy': '0.6958', 'epoch': '1.365'}
+{'loss': '1.364', 'grad_norm': '0.3596', 'learning_rate': '3.304e-05', 'entropy': '1.556', 'num_tokens': '2.409e+05', 'mean_token_accuracy': '0.6993', 'epoch': '1.383'}
+{'loss': '1.384', 'grad_norm': '0.3526', 'learning_rate': '3.214e-05', 'entropy': '1.545', 'num_tokens': '2.439e+05', 'mean_token_accuracy': '0.6964', 'epoch': '1.4'}
+{'loss': '1.368', 'grad_norm': '0.3533', 'learning_rate': '3.125e-05', 'entropy': '1.54', 'num_tokens': '2.47e+05', 'mean_token_accuracy': '0.698', 'epoch': '1.417'}
+{'loss': '1.383', 'grad_norm': '0.35', 'learning_rate': '3.036e-05', 'entropy': '1.551', 'num_tokens': '2.5e+05', 'mean_token_accuracy': '0.6982', 'epoch': '1.435'}
+{'loss': '1.371', 'grad_norm': '0.3513', 'learning_rate': '2.946e-05', 'entropy': '1.551', 'num_tokens': '2.531e+05', 'mean_token_accuracy': '0.6993', 'epoch': '1.452'}
+
+
+  0%|          | 0/52 [00:00<?, ?it/s]
+
+  4%|▍         | 2/52 [00:02<00:55,  1.10s/it]
+
+  6%|▌         | 3/52 [00:04<01:13,  1.51s/it]
+
+  8%|▊         | 4/52 [00:06<01:22,  1.72s/it]
+
+ 10%|▉         | 5/52 [00:08<01:26,  1.85s/it]
+
+ 12%|█▏        | 6/52 [00:10<01:28,  1.92s/it]
+
+ 13%|█▎        | 7/52 [00:12<01:29,  2.00s/it]
+
+ 15%|█▌        | 8/52 [00:14<01:28,  2.02s/it]
+
+ 17%|█▋        | 9/52 [00:16<01:25,  1.99s/it]
+
+ 19%|█▉        | 10/52 [00:18<01:22,  1.97s/it]
+
+ 21%|██        | 11/52 [00:20<01:20,  1.97s/it]
+
+ 23%|██▎       | 12/52 [00:22<01:17,  1.93s/it]
+
+ 25%|██▌       | 13/52 [00:24<01:13,  1.89s/it]
+
+ 27%|██▋       | 14/52 [00:26<01:11,  1.88s/it]
+
+ 29%|██▉       | 15/52 [00:27<01:09,  1.88s/it]
+
+ 31%|███       | 16/52 [00:29<01:07,  1.87s/it]
+
+ 33%|███▎      | 17/52 [00:31<01:05,  1.86s/it]
+
+ 35%|███▍      | 18/52 [00:33<01:03,  1.86s/it]
+
+ 37%|███▋      | 19/52 [00:35<01:01,  1.85s/it]
+
+ 38%|███▊      | 20/52 [00:37<00:59,  1.86s/it]
+
+ 40%|████      | 21/52 [00:39<00:57,  1.86s/it]
+
+ 42%|████▏     | 22/52 [00:40<00:55,  1.87s/it]
+
+ 44%|████▍     | 23/52 [00:42<00:54,  1.89s/it]
+
+ 46%|████▌     | 24/52 [00:44<00:52,  1.88s/it]
+
+ 48%|████▊     | 25/52 [00:47<00:55,  2.04s/it]
+
+ 50%|█████     | 26/52 [00:49<00:54,  2.09s/it]
+
+ 52%|█████▏    | 27/52 [00:51<00:50,  2.03s/it]
+
+ 54%|█████▍    | 28/52 [00:53<00:47,  1.97s/it]
+
+ 56%|█████▌    | 29/52 [00:54<00:44,  1.94s/it]
+
+ 58%|█████▊    | 30/52 [00:56<00:42,  1.92s/it]
+
+ 60%|█████▉    | 31/52 [00:58<00:40,  1.91s/it]
+
+ 62%|██████▏   | 32/52 [01:00<00:38,  1.91s/it]
+
+ 63%|██████▎   | 33/52 [01:02<00:35,  1.87s/it]
+
+ 65%|██████▌   | 34/52 [01:04<00:33,  1.88s/it]
+
+ 67%|██████▋   | 35/52 [01:06<00:32,  1.88s/it]
+
+ 69%|██████▉   | 36/52 [01:08<00:29,  1.86s/it]
+
+ 71%|███████   | 37/52 [01:09<00:27,  1.85s/it]
+
+ 73%|███████▎  | 38/52 [01:11<00:25,  1.85s/it]
+
+ 75%|███████▌  | 39/52 [01:13<00:23,  1.84s/it]
+
+ 77%|███████▋  | 40/52 [01:15<00:22,  1.85s/it]
+
+ 79%|███████▉  | 41/52 [01:17<00:20,  1.83s/it]
+
+ 81%|████████  | 42/52 [01:18<00:18,  1.82s/it]
+
+ 83%|████████▎ | 43/52 [01:20<00:16,  1.81s/it]
+
+ 85%|████████▍ | 44/52 [01:22<00:14,  1.81s/it]
+
+ 87%|████████▋ | 45/52 [01:24<00:12,  1.80s/it]
+
+ 88%|████████▊ | 46/52 [01:26<00:10,  1.83s/it]
+
+ 90%|█████████ | 47/52 [01:28<00:09,  1.82s/it]
+
+ 92%|█████████▏| 48/52 [01:29<00:07,  1.83s/it]
+
+ 94%|█████████▍| 49/52 [01:31<00:05,  1.83s/it]
+
+ 96%|█████████▌| 50/52 [01:33<00:03,  1.82s/it]
+
+ 98%|█████████▊| 51/52 [01:35<00:01,  1.82s/it]
+
+100%|██████████| 52/52 [01:37<00:00,  1.81s/it]
+                                                
+
+                                               
+
+ 72%|███████▏  | 84/116 [49:53<17:16, 32.40s/it]
+
+100%|██████████| 52/52 [01:37<00:00,  1.81s/it]
+
+                                               
+ 73%|███████▎  | 85/116 [50:23<31:50, 61.64s/it]
+                                                
+
+ 73%|███████▎  | 85/116 [50:23<31:50, 61.64s/it]
+ 74%|███████▍  | 86/116 [50:54<26:15, 52.53s/it]
+                                                
+
+ 74%|███████▍  | 86/116 [50:54<26:15, 52.53s/it]
+ 75%|███████▌  | 87/116 [51:27<22:27, 46.47s/it]
+                                                
+
+ 75%|███████▌  | 87/116 [51:27<22:27, 46.47s/it]
+ 76%|███████▌  | 88/116 [51:58<19:30, 41.79s/it]
+                                                
+
+ 76%|███████▌  | 88/116 [51:58<19:30, 41.79s/it]
+ 77%|███████▋  | 89/116 [52:31<17:43, 39.38s/it]
+                                                
+
+ 77%|███████▋  | 89/116 [52:31<17:43, 39.38s/it]
+ 78%|███████▊  | 90/116 [53:03<16:02, 37.00s/it]
+                                                
+
+ 78%|███████▊  | 90/116 [53:03<16:02, 37.00s/it]
+ 78%|███████▊  | 91/116 [53:34<14:42, 35.30s/it]
+                                                
+
+ 78%|███████▊  | 91/116 [53:34<14:42, 35.30s/it]
+ 79%|███████▉  | 92/116 [54:05<13:37, 34.08s/it]
+                                                
+
+ 79%|███████▉  | 92/116 [54:05<13:37, 34.08s/it]
+ 80%|████████  | 93/116 [54:38<12:53, 33.62s/it]
+                                                
+
+ 80%|████████  | 93/116 [54:38<12:53, 33.62s/it]
+ 81%|████████  | 94/116 [55:10<12:08, 33.13s/it]
+                                                
+
+ 81%|████████  | 94/116 [55:10<12:08, 33.13s/it]
+ 82%|████████▏ | 95/116 [55:42<11:26, 32.70s/it]
+                                                
+
+ 82%|████████▏ | 95/116 [55:42<11:26, 32.70s/it]
+ 83%|████████▎ | 96/116 [56:14<10:49, 32.48s/it]
+                                                
+
+ 83%|████████▎ | 96/116 [56:14<10:49, 32.48s/it]
+ 84%|████████▎ | 97/116 [56:45<10:12, 32.22s/it]
+                                                
+
+ 84%|████████▎ | 97/116 [56:45<10:12, 32.22s/it]
+ 84%|████████▍ | 98/116 [57:20<09:53, 32.95s/it]
+                                                
+
+ 84%|████████▍ | 98/116 [57:20<09:53, 32.95s/it]
+ 85%|████████▌ | 99/116 [57:52<09:15, 32.67s/it]
+                                                
+
+ 85%|████████▌ | 99/116 [57:52<09:15, 32.67s/it]
+ 86%|████████▌ | 100/116 [58:25<08:44, 32.81s/it]
+                                                 
+
+ 86%|████████▌ | 100/116 [58:25<08:44, 32.81s/it]
+ 87%|████████▋ | 101/116 [58:58<08:12, 32.86s/it]
+                                                 
+
+ 87%|████████▋ | 101/116 [58:58<08:12, 32.86s/it]
+ 88%|████████▊ | 102/116 [59:31<07:40, 32.86s/it]
+                                                 
+
+ 88%|████████▊ | 102/116 [59:31<07:40, 32.86s/it]
+ 89%|████████▉ | 103/116 [1:00:01<06:56, 32.05s/it]
+                                                   
+
+ 89%|████████▉ | 103/116 [1:00:01<06:56, 32.05s/it]
+ 90%|████████▉ | 104/116 [1:00:33<06:24, 32.05s/it]
+                                                   
+
+ 90%|████████▉ | 104/116 [1:00:33<06:24, 32.05s/it]
+ 91%|█████████ | 105/116 [1:01:08<06:00, 32.76s/it]
+                                                   
+
+ 91%|█████████ | 105/116 [1:01:08<06:00, 32.76s/it]
+ 91%|█████████▏| 106/116 [1:01:39<05:24, 32.48s/it]
+                                                   
+
+ 91%|█████████▏| 106/116 [1:01:39<05:24, 32.48s/it]
+ 92%|█████████▏| 107/116 [1:02:12<04:54, 32.67s/it]
+                                                   
+
+ 92%|█████████▏| 107/116 [1:02:12<04:54, 32.67s/it]
+ 93%|█████████▎| 108/116 [1:02:43<04:16, 32.08s/it]
+                                                   
+
+ 93%|█████████▎| 108/116 [1:02:43<04:16, 32.08s/it]
+ 94%|█████████▍| 109/116 [1:03:16<03:45, 32.18s/it]
+                                                   
+
+ 94%|█████████▍| 109/116 [1:03:16<03:45, 32.18s/it]
+ 95%|█████████▍| 110/116 [1:03:47<03:11, 31.97s/it]
+                                                   
+
+ 95%|█████████▍| 110/116 [1:03:47<03:11, 31.97s/it]
+ 96%|█████████▌| 111/116 [1:04:20<02:40, 32.14s/it]
+                                                   
+
+ 96%|█████████▌| 111/116 [1:04:20<02:40, 32.14s/it]
+ 97%|█████████▋| 112/116 [1:04:52<02:08, 32.14s/it]
+                                                   
+
+ 97%|█████████▋| 112/116 [1:04:52<02:08, 32.14s/it]{'eval_loss': '1.337', 'eval_runtime': '99.21', 'eval_samples_per_second': '0.524', 'eval_steps_per_second': '0.524', 'eval_entropy': '1.529', 'eval_num_tokens': '2.531e+05', 'eval_mean_token_accuracy': '0.7008', 'epoch': '1.452'}
+{'loss': '1.339', 'grad_norm': '0.3704', 'learning_rate': '2.857e-05', 'entropy': '1.542', 'num_tokens': '2.56e+05', 'mean_token_accuracy': '0.6978', 'epoch': '1.47'}
+{'loss': '1.344', 'grad_norm': '0.3635', 'learning_rate': '2.768e-05', 'entropy': '1.534', 'num_tokens': '2.591e+05', 'mean_token_accuracy': '0.7019', 'epoch': '1.487'}
+{'loss': '1.345', 'grad_norm': '0.3585', 'learning_rate': '2.679e-05', 'entropy': '1.537', 'num_tokens': '2.621e+05', 'mean_token_accuracy': '0.6942', 'epoch': '1.504'}
+{'loss': '1.331', 'grad_norm': '0.3683', 'learning_rate': '2.589e-05', 'entropy': '1.528', 'num_tokens': '2.651e+05', 'mean_token_accuracy': '0.7035', 'epoch': '1.522'}
+{'loss': '1.33', 'grad_norm': '0.3597', 'learning_rate': '2.5e-05', 'entropy': '1.521', 'num_tokens': '2.682e+05', 'mean_token_accuracy': '0.7037', 'epoch': '1.539'}
+{'loss': '1.302', 'grad_norm': '0.372', 'learning_rate': '2.411e-05', 'entropy': '1.497', 'num_tokens': '2.712e+05', 'mean_token_accuracy': '0.7037', 'epoch': '1.557'}
+{'loss': '1.319', 'grad_norm': '0.3674', 'learning_rate': '2.321e-05', 'entropy': '1.513', 'num_tokens': '2.742e+05', 'mean_token_accuracy': '0.6993', 'epoch': '1.574'}
+{'loss': '1.32', 'grad_norm': '0.3708', 'learning_rate': '2.232e-05', 'entropy': '1.522', 'num_tokens': '2.772e+05', 'mean_token_accuracy': '0.6944', 'epoch': '1.591'}
+{'loss': '1.33', 'grad_norm': '0.3596', 'learning_rate': '2.143e-05', 'entropy': '1.507', 'num_tokens': '2.803e+05', 'mean_token_accuracy': '0.6984', 'epoch': '1.609'}
+{'loss': '1.285', 'grad_norm': '0.3709', 'learning_rate': '2.054e-05', 'entropy': '1.485', 'num_tokens': '2.833e+05', 'mean_token_accuracy': '0.7081', 'epoch': '1.626'}
+{'loss': '1.282', 'grad_norm': '0.3686', 'learning_rate': '1.964e-05', 'entropy': '1.482', 'num_tokens': '2.864e+05', 'mean_token_accuracy': '0.7009', 'epoch': '1.643'}
+{'loss': '1.285', 'grad_norm': '0.3653', 'learning_rate': '1.875e-05', 'entropy': '1.481', 'num_tokens': '2.894e+05', 'mean_token_accuracy': '0.7066', 'epoch': '1.661'}
+{'loss': '1.285', 'grad_norm': '0.3654', 'learning_rate': '1.786e-05', 'entropy': '1.484', 'num_tokens': '2.925e+05', 'mean_token_accuracy': '0.6989', 'epoch': '1.678'}
+{'loss': '1.276', 'grad_norm': '0.3708', 'learning_rate': '1.696e-05', 'entropy': '1.474', 'num_tokens': '2.955e+05', 'mean_token_accuracy': '0.7062', 'epoch': '1.696'}
+{'loss': '1.261', 'grad_norm': '0.3831', 'learning_rate': '1.607e-05', 'entropy': '1.482', 'num_tokens': '2.985e+05', 'mean_token_accuracy': '0.7098', 'epoch': '1.713'}
+{'loss': '1.278', 'grad_norm': '0.3693', 'learning_rate': '1.518e-05', 'entropy': '1.485', 'num_tokens': '3.015e+05', 'mean_token_accuracy': '0.7034', 'epoch': '1.73'}
+{'loss': '1.277', 'grad_norm': '0.3711', 'learning_rate': '1.429e-05', 'entropy': '1.472', 'num_tokens': '3.046e+05', 'mean_token_accuracy': '0.7034', 'epoch': '1.748'}
+{'loss': '1.272', 'grad_norm': '0.3784', 'learning_rate': '1.339e-05', 'entropy': '1.475', 'num_tokens': '3.077e+05', 'mean_token_accuracy': '0.7076', 'epoch': '1.765'}
+{'loss': '1.25', 'grad_norm': '0.3946', 'learning_rate': '1.25e-05', 'entropy': '1.479', 'num_tokens': '3.106e+05', 'mean_token_accuracy': '0.7072', 'epoch': '1.783'}
+{'loss': '1.249', 'grad_norm': '0.3854', 'learning_rate': '1.161e-05', 'entropy': '1.458', 'num_tokens': '3.136e+05', 'mean_token_accuracy': '0.7067', 'epoch': '1.8'}
+{'loss': '1.259', 'grad_norm': '0.3789', 'learning_rate': '1.071e-05', 'entropy': '1.472', 'num_tokens': '3.166e+05', 'mean_token_accuracy': '0.711', 'epoch': '1.817'}
+{'loss': '1.235', 'grad_norm': '0.382', 'learning_rate': '9.821e-06', 'entropy': '1.448', 'num_tokens': '3.197e+05', 'mean_token_accuracy': '0.7144', 'epoch': '1.835'}
+{'loss': '1.268', 'grad_norm': '0.3723', 'learning_rate': '8.929e-06', 'entropy': '1.481', 'num_tokens': '3.227e+05', 'mean_token_accuracy': '0.7034', 'epoch': '1.852'}
+{'loss': '1.227', 'grad_norm': '0.3826', 'learning_rate': '8.036e-06', 'entropy': '1.447', 'num_tokens': '3.257e+05', 'mean_token_accuracy': '0.7107', 'epoch': '1.87'}
+{'loss': '1.24', 'grad_norm': '0.3853', 'learning_rate': '7.143e-06', 'entropy': '1.454', 'num_tokens': '3.288e+05', 'mean_token_accuracy': '0.7155', 'epoch': '1.887'}
+{'loss': '1.238', 'grad_norm': '0.3807', 'learning_rate': '6.25e-06', 'entropy': '1.453', 'num_tokens': '3.318e+05', 'mean_token_accuracy': '0.7101', 'epoch': '1.904'}
+{'loss': '1.23', 'grad_norm': '0.3776', 'learning_rate': '5.357e-06', 'entropy': '1.439', 'num_tokens': '3.348e+05', 'mean_token_accuracy': '0.7141', 'epoch': '1.922'}
+{'loss': '1.257', 'grad_norm': '0.378', 'learning_rate': '4.464e-06', 'entropy': '1.459', 'num_tokens': '3.38e+05', 'mean_token_accuracy': '0.717', 'epoch': '1.939'}
+
+
+  0%|          | 0/52 [00:00<?, ?it/s]
+
+  4%|▍         | 2/52 [00:02<00:51,  1.04s/it]
+
+  6%|▌         | 3/52 [00:04<01:13,  1.50s/it]
+
+  8%|▊         | 4/52 [00:06<01:20,  1.69s/it]
+
+ 10%|▉         | 5/52 [00:08<01:28,  1.89s/it]
+
+ 12%|█▏        | 6/52 [00:10<01:34,  2.06s/it]
+
+ 13%|█▎        | 7/52 [00:13<01:35,  2.11s/it]
+
+ 15%|█▌        | 8/52 [00:15<01:31,  2.08s/it]
+
+ 17%|█▋        | 9/52 [00:17<01:28,  2.05s/it]
+
+ 19%|█▉        | 10/52 [00:19<01:24,  2.02s/it]
+
+ 21%|██        | 11/52 [00:20<01:20,  1.96s/it]
+
+ 23%|██▎       | 12/52 [00:22<01:17,  1.94s/it]
+
+ 25%|██▌       | 13/52 [00:24<01:15,  1.93s/it]
+
+ 27%|██▋       | 14/52 [00:26<01:12,  1.91s/it]
+
+ 29%|██▉       | 15/52 [00:28<01:10,  1.91s/it]
+
+ 31%|███       | 16/52 [00:30<01:08,  1.89s/it]
+
+ 33%|███▎      | 17/52 [00:32<01:05,  1.88s/it]
+
+ 35%|███▍      | 18/52 [00:34<01:04,  1.89s/it]
+
+ 37%|███▋      | 19/52 [00:35<01:02,  1.89s/it]
+
+ 38%|███▊      | 20/52 [00:37<01:00,  1.88s/it]
+
+ 40%|████      | 21/52 [00:39<00:58,  1.87s/it]
+
+ 42%|████▏     | 22/52 [00:41<00:55,  1.84s/it]
+
+ 44%|████▍     | 23/52 [00:43<00:53,  1.85s/it]
+
+ 46%|████▌     | 24/52 [00:45<00:51,  1.84s/it]
+
+ 48%|████▊     | 25/52 [00:46<00:49,  1.84s/it]
+
+ 50%|█████     | 26/52 [00:48<00:47,  1.83s/it]
+
+ 52%|█████▏    | 27/52 [00:50<00:46,  1.84s/it]
+
+ 54%|█████▍    | 28/52 [00:52<00:43,  1.83s/it]
+
+ 56%|█████▌    | 29/52 [00:54<00:41,  1.81s/it]
+
+ 58%|█████▊    | 30/52 [00:55<00:39,  1.80s/it]
+
+ 60%|█████▉    | 31/52 [00:57<00:37,  1.80s/it]
+
+ 62%|██████▏   | 32/52 [00:59<00:36,  1.82s/it]
+
+ 63%|██████▎   | 33/52 [01:01<00:34,  1.82s/it]
+
+ 65%|██████▌   | 34/52 [01:03<00:32,  1.83s/it]
+
+ 67%|██████▋   | 35/52 [01:05<00:30,  1.82s/it]
+
+ 69%|██████▉   | 36/52 [01:07<00:30,  1.92s/it]
+
+ 71%|███████   | 37/52 [01:10<00:32,  2.19s/it]
+
+ 73%|███████▎  | 38/52 [01:12<00:30,  2.20s/it]
+
+ 75%|███████▌  | 39/52 [01:14<00:27,  2.13s/it]
+
+ 77%|███████▋  | 40/52 [01:16<00:24,  2.05s/it]
+
+ 79%|███████▉  | 41/52 [01:17<00:21,  1.98s/it]
+
+ 81%|████████  | 42/52 [01:19<00:19,  1.92s/it]
+
+ 83%|████████▎ | 43/52 [01:21<00:16,  1.88s/it]
+
+ 85%|████████▍ | 44/52 [01:23<00:14,  1.86s/it]
+
+ 87%|████████▋ | 45/52 [01:25<00:12,  1.84s/it]
+
+ 88%|████████▊ | 46/52 [01:26<00:10,  1.81s/it]
+
+ 90%|█████████ | 47/52 [01:28<00:09,  1.82s/it]
+
+ 92%|█████████▏| 48/52 [01:30<00:07,  1.81s/it]
+
+ 94%|█████████▍| 49/52 [01:32<00:05,  1.80s/it]
+
+ 96%|█████████▌| 50/52 [01:34<00:03,  1.79s/it]
+
+ 98%|█████████▊| 51/52 [01:35<00:01,  1.78s/it]
+
+100%|██████████| 52/52 [01:37<00:00,  1.75s/it]
+                                                   
+
+                                               
+
+ 97%|█████████▋| 112/116 [1:06:31<02:08, 32.14s/it]
+
+100%|██████████| 52/52 [01:37<00:00,  1.75s/it]
+
+                                               
+ 97%|█████████▋| 113/116 [1:07:01<03:03, 61.26s/it]
+                                                   
+
+ 97%|█████████▋| 113/116 [1:07:01<03:03, 61.26s/it]
+ 98%|█████████▊| 114/116 [1:07:33<01:45, 52.64s/it]
+                                                   
+
+ 98%|█████████▊| 114/116 [1:07:33<01:45, 52.64s/it]
+ 99%|█████████▉| 115/116 [1:08:06<00:46, 46.62s/it]
+                                                   
+
+ 99%|█████████▉| 115/116 [1:08:06<00:46, 46.62s/it]
+100%|██████████| 116/116 [1:08:22<00:00, 37.52s/it]
+                                                   
+
+100%|██████████| 116/116 [1:08:22<00:00, 37.52s/it]{'eval_loss': '1.227', 'eval_runtime': '99.61', 'eval_samples_per_second': '0.522', 'eval_steps_per_second': '0.522', 'eval_entropy': '1.447', 'eval_num_tokens': '3.38e+05', 'eval_mean_token_accuracy': '0.7102', 'epoch': '1.939'}
+{'loss': '1.222', 'grad_norm': '0.3988', 'learning_rate': '3.571e-06', 'entropy': '1.453', 'num_tokens': '3.409e+05', 'mean_token_accuracy': '0.7062', 'epoch': '1.957'}
+{'loss': '1.236', 'grad_norm': '0.3919', 'learning_rate': '2.679e-06', 'entropy': '1.469', 'num_tokens': '3.438e+05', 'mean_token_accuracy': '0.705', 'epoch': '1.974'}
+{'loss': '1.262', 'grad_norm': '0.3766', 'learning_rate': '1.786e-06', 'entropy': '1.462', 'num_tokens': '3.469e+05', 'mean_token_accuracy': '0.7158', 'epoch': '1.991'}
+{'loss': '1.255', 'grad_norm': '0.3805', 'learning_rate': '8.929e-07', 'entropy': '1.467', 'num_tokens': '3.485e+05', 'mean_token_accuracy': '0.7127', 'epoch': '2'}
+
+
+  0%|          | 0/52 [00:00<?, ?it/s]
+
+  4%|▍         | 2/52 [00:02<00:50,  1.00s/it]
+
+  6%|▌         | 3/52 [00:04<01:11,  1.45s/it]
+
+  8%|▊         | 4/52 [00:06<01:19,  1.66s/it]
+
+ 10%|▉         | 5/52 [00:08<01:24,  1.79s/it]
+
+ 12%|█▏        | 6/52 [00:10<01:26,  1.87s/it]
+
+ 13%|█▎        | 7/52 [00:12<01:27,  1.95s/it]
+
+ 15%|█▌        | 8/52 [00:14<01:26,  1.96s/it]
+
+ 17%|█▋        | 9/52 [00:16<01:23,  1.95s/it]
+
+ 19%|█▉        | 10/52 [00:18<01:22,  1.96s/it]
+
+ 21%|██        | 11/52 [00:20<01:19,  1.93s/it]
+
+ 23%|██▎       | 12/52 [00:21<01:16,  1.91s/it]
+
+ 25%|██▌       | 13/52 [00:23<01:13,  1.90s/it]
+
+ 27%|██▋       | 14/52 [00:25<01:11,  1.89s/it]
+
+ 29%|██▉       | 15/52 [00:27<01:10,  1.90s/it]
+
+ 31%|███       | 16/52 [00:29<01:07,  1.88s/it]
+
+ 33%|███▎      | 17/52 [00:31<01:05,  1.87s/it]
+
+ 35%|███▍      | 18/52 [00:33<01:03,  1.88s/it]
+
+ 37%|███▋      | 19/52 [00:34<01:01,  1.85s/it]
+
+ 38%|███▊      | 20/52 [00:36<01:01,  1.91s/it]
+
+ 40%|████      | 21/52 [00:39<01:03,  2.05s/it]
+
+ 42%|████▏     | 22/52 [00:41<01:00,  2.01s/it]
+
+ 44%|████▍     | 23/52 [00:43<00:56,  1.96s/it]
+
+ 46%|████▌     | 24/52 [00:44<00:54,  1.93s/it]
+
+ 48%|████▊     | 25/52 [00:46<00:50,  1.89s/it]
+
+ 50%|█████     | 26/52 [00:48<00:48,  1.85s/it]
+
+ 52%|█████▏    | 27/52 [00:50<00:45,  1.82s/it]
+
+ 54%|█████▍    | 28/52 [00:52<00:43,  1.83s/it]
+
+ 56%|█████▌    | 29/52 [00:53<00:42,  1.83s/it]
+
+ 58%|█████▊    | 30/52 [00:55<00:40,  1.83s/it]
+
+ 60%|█████▉    | 31/52 [00:57<00:38,  1.81s/it]
+
+ 62%|██████▏   | 32/52 [00:59<00:36,  1.80s/it]
+
+ 63%|██████▎   | 33/52 [01:01<00:33,  1.79s/it]
+
+ 65%|██████▌   | 34/52 [01:02<00:32,  1.78s/it]
+
+ 67%|██████▋   | 35/52 [01:04<00:30,  1.77s/it]
+
+ 69%|██████▉   | 36/52 [01:06<00:28,  1.78s/it]
+
+ 71%|███████   | 37/52 [01:08<00:26,  1.78s/it]
+
+ 73%|███████▎  | 38/52 [01:09<00:24,  1.76s/it]
+
+ 75%|███████▌  | 39/52 [01:11<00:22,  1.75s/it]
+
+ 77%|███████▋  | 40/52 [01:13<00:21,  1.75s/it]
+
+ 79%|███████▉  | 41/52 [01:15<00:19,  1.77s/it]
+
+ 81%|████████  | 42/52 [01:17<00:17,  1.79s/it]
+
+ 83%|████████▎ | 43/52 [01:18<00:16,  1.79s/it]
+
+ 85%|████████▍ | 44/52 [01:20<00:14,  1.78s/it]
+
+ 87%|████████▋ | 45/52 [01:22<00:12,  1.80s/it]
+
+ 88%|████████▊ | 46/52 [01:24<00:10,  1.82s/it]
+
+ 90%|█████████ | 47/52 [01:26<00:09,  1.81s/it]
+
+ 92%|█████████▏| 48/52 [01:27<00:07,  1.80s/it]
+
+ 94%|█████████▍| 49/52 [01:29<00:05,  1.79s/it]
+
+ 96%|█████████▌| 50/52 [01:31<00:03,  1.79s/it]
+
+ 98%|█████████▊| 51/52 [01:33<00:01,  1.82s/it]
+
+100%|██████████| 52/52 [01:35<00:00,  1.81s/it]
+                                                   
+
+                                               
+
+100%|██████████| 116/116 [1:10:00<00:00, 37.52s/it]
+
+100%|██████████| 52/52 [01:35<00:00,  1.81s/it]
+
+                                               
+                                                   
+
+100%|██████████| 116/116 [1:10:00<00:00, 37.52s/it]
+100%|██████████| 116/116 [1:10:00<00:00, 36.21s/it]
+{'eval_loss': '1.225', 'eval_runtime': '97.16', 'eval_samples_per_second': '0.535', 'eval_steps_per_second': '0.535', 'eval_entropy': '1.445', 'eval_num_tokens': '3.485e+05', 'eval_mean_token_accuracy': '0.7107', 'epoch': '2'}
+{'train_runtime': '4201', 'train_samples_per_second': '0.219', 'train_steps_per_second': '0.028', 'train_loss': '1.65', 'epoch': '2'}
+checkpoint: out/runs/mindx_fallback_qwen3_1_5b_cpu_real/checkpoint