Skip to content

Commit

Permalink
CU-8694vxcyx MedCAT 1.12 support (#22)
Browse files Browse the repository at this point in the history 
* CU-8693v4mx1: Update ICD10 and OPCS4 mappings in SNOMED CDB preprocessing

* CU-8694vxcyx: Remove deprecated method usage and reference

* CU-8694vxcyx: Bump medcat requirement to 1.12
  • Loading branch information
@mart-r
mart-r authored Jun 25, 2024
1 parent c61b47a commit 563e156
Show file tree
Hide file tree
Showing 3 changed files with 34 additions and 34 deletions.
50 changes: 25 additions & 25 deletions notebooks/introductory/Part_4_2_Supervised_Training_and_Meta_annotations.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13799,7 +13799,7 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>To fine-tune the existing models we use the <code>train_supervised()</code> method from MedCAT. The method has the following options:</p>
<p>To fine-tune the existing models we use the <code>train_supervised_from_json()</code> method from MedCAT. The method has the following options:</p>
<hr>
<p><code>data_path</code> - Path to the JSON file exported from MedCATtrainer</p>
<p><code>reset_cui_count</code> - Each cui has an internal counter that is used for weight decay, this will reset it.</p>
Expand All @@ -13823,11 +13823,11 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
<div class="prompt input_prompt">In&nbsp;[11]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">cat</span><span class="o">.</span><span class="n">train_supervised</span><span class="p">(</span><span class="n">data_path</span><span class="o">=</span><span class="n">DATA_DIR</span> <span class="o">+</span> <span class="s2">&quot;MedCAT_Export.json&quot;</span><span class="p">,</span>
<span class="n">nepochs</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span>
<span class="n">reset_cui_count</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
<span class="n">print_stats</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
<span class="n">use_filters</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">cat</span><span class="o">.</span><span class="n">train_supervised_from_json</span><span class="p">(</span><span class="n">data_path</span><span class="o">=</span><span class="n">DATA_DIR</span> <span class="o">+</span> <span class="s2">&quot;MedCAT_Export.json&quot;</span><span class="p">,</span>
<span class="n">nepochs</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span>
<span class="n">reset_cui_count</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
<span class="n">print_stats</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
<span class="n">use_filters</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</pre></div>

</div>
Expand Down Expand Up @@ -13859,10 +13859,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla



<div id="6ae0b7ea-bdeb-4d3f-b83c-3342c9b13cd3"></div>
<div id="97a59b96-c389-409a-8457-732acfff3f84"></div>
<div class="output_subarea output_widget_view ">
<script type="text/javascript">
var element = $('#6ae0b7ea-bdeb-4d3f-b83c-3342c9b13cd3');
var element = $('#97a59b96-c389-409a-8457-732acfff3f84');
</script>
<script type="application/vnd.jupyter.widget-view+json">
{"model_id": "6fd10f1692234019836a7b40e83b56dd", "version_major": 2, "version_minor": 0}
Expand All @@ -13881,10 +13881,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla



<div id="226f4eb5-22f1-43d4-83ae-4f42f75a13ba"></div>
<div id="949bb97a-d72f-4166-b47d-d4ffe721894c"></div>
<div class="output_subarea output_widget_view ">
<script type="text/javascript">
var element = $('#226f4eb5-22f1-43d4-83ae-4f42f75a13ba');
var element = $('#949bb97a-d72f-4166-b47d-d4ffe721894c');
</script>
<script type="application/vnd.jupyter.widget-view+json">
{"model_id": "9a5ab9cfecc242b7aaf0f140e87bdde6", "version_major": 2, "version_minor": 0}
Expand Down Expand Up @@ -13963,10 +13963,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla



<div id="04326482-ffec-4b5c-8633-f5562a193720"></div>
<div id="39e15e56-fb47-4b4b-bc54-840980738477"></div>
<div class="output_subarea output_widget_view ">
<script type="text/javascript">
var element = $('#04326482-ffec-4b5c-8633-f5562a193720');
var element = $('#39e15e56-fb47-4b4b-bc54-840980738477');
</script>
<script type="application/vnd.jupyter.widget-view+json">
{"model_id": "434496e448984f55925d22fad0349ada", "version_major": 2, "version_minor": 0}
Expand All @@ -13985,10 +13985,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla



<div id="f6af9754-3d34-43f0-931c-ae2d0882d7b7"></div>
<div id="05772180-dbe8-451b-a09a-c4435304092d"></div>
<div class="output_subarea output_widget_view ">
<script type="text/javascript">
var element = $('#f6af9754-3d34-43f0-931c-ae2d0882d7b7');
var element = $('#05772180-dbe8-451b-a09a-c4435304092d');
</script>
<script type="application/vnd.jupyter.widget-view+json">
{"model_id": "f7d1803b3c6c4197b6612c5fdf189746", "version_major": 2, "version_minor": 0}
Expand All @@ -14007,10 +14007,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla



<div id="fdc1a5ce-011a-4f3f-a5e2-ac061f13c471"></div>
<div id="4a031d0b-0138-4bde-ab47-a769a96bec8d"></div>
<div class="output_subarea output_widget_view ">
<script type="text/javascript">
var element = $('#fdc1a5ce-011a-4f3f-a5e2-ac061f13c471');
var element = $('#4a031d0b-0138-4bde-ab47-a769a96bec8d');
</script>
<script type="application/vnd.jupyter.widget-view+json">
{"model_id": "c8d633f579de438a916d9ef3de9d8fe0", "version_major": 2, "version_minor": 0}
Expand All @@ -14029,10 +14029,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla



<div id="c247a603-12d5-4cff-b8df-58babf41c20c"></div>
<div id="9b3536c0-957e-4873-97cf-0e5b64fd2cb4"></div>
<div class="output_subarea output_widget_view ">
<script type="text/javascript">
var element = $('#c247a603-12d5-4cff-b8df-58babf41c20c');
var element = $('#9b3536c0-957e-4873-97cf-0e5b64fd2cb4');
</script>
<script type="application/vnd.jupyter.widget-view+json">
{"model_id": "de6c01c6983041e2b972f6008caefaea", "version_major": 2, "version_minor": 0}
Expand All @@ -14051,10 +14051,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla



<div id="fe3fd5be-68e4-440a-9093-c4a51d88b939"></div>
<div id="0c50692e-679a-4580-a98c-d9b7c5a7b647"></div>
<div class="output_subarea output_widget_view ">
<script type="text/javascript">
var element = $('#fe3fd5be-68e4-440a-9093-c4a51d88b939');
var element = $('#0c50692e-679a-4580-a98c-d9b7c5a7b647');
</script>
<script type="application/vnd.jupyter.widget-view+json">
{"model_id": "05132c907a874fe2a2eb9cb6c81da3b3", "version_major": 2, "version_minor": 0}
Expand Down Expand Up @@ -17502,10 +17502,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla



<div id="993b7d0d-31c6-4cb9-8a67-229e48373814"></div>
<div id="0831575f-3f80-433f-8e19-3ac896196ba2"></div>
<div class="output_subarea output_widget_view ">
<script type="text/javascript">
var element = $('#993b7d0d-31c6-4cb9-8a67-229e48373814');
var element = $('#0831575f-3f80-433f-8e19-3ac896196ba2');
</script>
<script type="application/vnd.jupyter.widget-view+json">
{"model_id": "00325922360c45009329d82ed6420f16", "version_major": 2, "version_minor": 0}
Expand All @@ -17524,10 +17524,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla



<div id="cba1d441-64a6-4dcc-ab5a-5ae744aaff24"></div>
<div id="09e4a13d-2327-4e85-87d4-3d2777ffc588"></div>
<div class="output_subarea output_widget_view ">
<script type="text/javascript">
var element = $('#cba1d441-64a6-4dcc-ab5a-5ae744aaff24');
var element = $('#09e4a13d-2327-4e85-87d4-3d2777ffc588');
</script>
<script type="application/vnd.jupyter.widget-view+json">
{"model_id": "d48e2f4d6dd3467fb3f17e0244b0e361", "version_major": 2, "version_minor": 0}
Expand Down Expand Up @@ -17926,7 +17926,7 @@ <h3 id="Train-MetaCAT">Train MetaCAT<a class="anchor-link" href="#Train-MetaCAT"
<div class="prompt input_prompt">In&nbsp;[19]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">mc</span><span class="o">.</span><span class="n">train</span><span class="p">(</span><span class="n">json_path</span><span class="o">=</span> <span class="n">DATA_DIR</span><span class="o">+</span><span class="s2">&quot;MedCAT_Export.json&quot;</span><span class="p">,</span> <span class="n">save_dir_path</span><span class="o">=</span><span class="s1">&#39;status&#39;</span><span class="p">)</span>
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">mc</span><span class="o">.</span><span class="n">train_from_json</span><span class="p">(</span><span class="n">json_path</span><span class="o">=</span> <span class="n">DATA_DIR</span><span class="o">+</span><span class="s2">&quot;MedCAT_Export.json&quot;</span><span class="p">,</span> <span class="n">save_dir_path</span><span class="o">=</span><span class="s1">&#39;status&#39;</span><span class="p">)</span>
<span class="c1"># Saving the model this way will only save the model epoch with the best performance</span>
</pre></div>

Expand Down
14 changes: 7 additions & 7 deletions notebooks/introductory/Part_4_2_Supervised_Training_and_Meta_annotations.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -664,7 +664,7 @@
"id": "OTR2bcSZDP5V"
},
"source": [
"To fine-tune the existing models we use the `train_supervised()` method from MedCAT. The method has the following options:\n",
"To fine-tune the existing models we use the `train_supervised_from_json()` method from MedCAT. The method has the following options:\n",
"\n",
"\n",
"---\n",
Expand Down Expand Up @@ -4353,11 +4353,11 @@
}
],
"source": [
"cat.train_supervised(data_path=DATA_DIR + \"MedCAT_Export.json\", \n",
" nepochs=1,\n",
" reset_cui_count=False,\n",
" print_stats=True, \n",
" use_filters=True) "
"cat.train_supervised_from_json(data_path=DATA_DIR + \"MedCAT_Export.json\", \n",
" nepochs=1,\n",
" reset_cui_count=False,\n",
" print_stats=True, \n",
" use_filters=True) "
]
},
{
Expand Down Expand Up @@ -6001,7 +6001,7 @@
}
],
"source": [
"mc.train(json_path= DATA_DIR+\"MedCAT_Export.json\", save_dir_path='status')\n",
"mc.train_from_json(json_path= DATA_DIR+\"MedCAT_Export.json\", save_dir_path='status')\n",
"# Saving the model this way will only save the model epoch with the best performance"
]
},
Expand Down
4 changes: 2 additions & 2 deletions requirements-dev.txt
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
medcat~=1.10.0
medcat~=1.12.0
pandas<2.0.0
seaborn~=0.11.2
pytest-xdist~=2.5.0
nbmake<1.4
nbconvert<6
jinja2<=3.0
seaborn
seaborn

0 comments on commit 563e156

Please sign in to comment.