2026-10-12 https://api.researchdata.se/oai-pmh

oai:researchdata.se:doi-10-23695-aryw-nh78/0 2020-06-29 subject:ssif:10208 subject:ssif:102 subject:ssif:1 principal:slug:university-of-gothenburg

info:eu-repo/semantics/other http://purl.org/dc/dcmitype/Dataset https://doi.org/10.23695/ARYW-NH78 POS-tagging model: Marmot Ordklasstaggningsmodell: Marmot https://ror.org/05qhvy459 Natural Language Processing Språkbehandling och datorlingvistik Model Marmot is a part-of-speech tagger that performs for Swedish somewhat worse than state-of-the-art neural models, but still yields very good results, works much faster and does not require a GPU. We provide two models for Marmot. marmot_eval is trained on SUC3 and Talbanken_SBX_dev, using Saldo as dictionary. The advantage of this model is that it can be evaluated, using Talbanken_SBX_test or SIC2. The evaluation results are reported in the table below. Test set Exact match POS MSD Talbanken_SBX_test 0.973 0.982 0.988 SIC2 0.921 0.934 0.958 Read more about the evaluation here. marmot_full is trained on SUC3 + Talbanken_SBX_test + Talbanken_SBX_dev + SIC2 (with Saldo as dictionary). We cannot evaluate the performance of this model, but we expect it to perform better than marmot_eval, or at least not worse. Tagging and training Download Marmot and the necessary dependencies. Download SALDO (converted to the necessary format) here. Download our scripts from this repository. The scripts use a tab-separated three-column format: token, POS (without MSD), MSD. Use conllu_to_tab.rb to convert CONLL(U) to the two-column format (install Ruby 1.9+ and run ruby conllu_to_tab 2 n, where n is the number of the column you want to use (if you are converting our CONLLU files, use 4). Run ruby convert_col2_to_marmot.rb to convert the resulting col2 file to Marmot's col3). Tagging Use java -cp marmot.jar marmot.morph.cmd.Annotator --model-file model_name.marmot --test-file form-index=0,test_corpus.col1 --pred-file output_name.conll to tag a corpus using a pretrained model. The output corpus will be in a CONLL format with a somewhat unusual order of columns, use convert_marmot_to_conllu.rb to convert it to a usual CONLLU. Training your own models Run Marmot: java -Xmx5G -cp marmot.jar marmot.morph.cmd.Trainer -train-file form-index=0,tag-index=1,morph-index=2,corpus.col3 -tag-morph true -model-file model_name.marmot subtag-separator "." -type-dict saldo_marmot.txt,indexes=[2,3] Model Marmot is a part-of-speech tagger that performs for Swedish somewhat worse than state-of-the-art neural models, but still yields very good results, works much faster and does not require a GPU. We provide two models for Marmot. marmot_eval is trained on SUC3 and Talbanken_SBX_dev, using Saldo as dictionary. The advantage of this model is that it can be evaluated, using Talbanken_SBX_test or SIC2. The evaluation results are reported in the table below. Test set Exact match POS MSD Talbanken_SBX_test 0.973 0.982 0.988 SIC2 0.921 0.934 0.958 Read more about the evaluation here. marmot_full is trained on SUC3 + Talbanken_SBX_test + Talbanken_SBX_dev + SIC2 (with Saldo as dictionary). We cannot evaluate the performance of this model, but we expect it to perform better than marmot_eval, or at least not worse. Tagging and training Download Marmot and the necessary dependencies. Download SALDO (converted to the necessary format) here. Download our scripts from this repository. The scripts use a tab-separated three-column format: token, POS (without MSD), MSD. Use conllu_to_tab.rb to convert CONLL(U) to the two-column format (install Ruby 1.9+ and run ruby conllu_to_tab 2 n, where n is the number of the column you want to use (if you are converting our CONLLU files, use 4). Run ruby convert_col2_to_marmot.rb to convert the resulting col2 file to Marmot's col3). Tagging Use java -cp marmot.jar marmot.morph.cmd.Annotator --model-file model_name.marmot --test-file form-index=0,test_corpus.col1 --pred-file output_name.conll to tag a corpus using a pretrained model. The output corpus will be in a CONLL format with a somewhat unusual order of columns, use convert_marmot_to_conllu.rb to convert it to a usual CONLLU. Training your own models Run Marmot: java -Xmx5G -cp marmot.jar marmot.morph.cmd.Trainer -train-file form-index=0,tag-index=1,morph-index=2,corpus.col3 -tag-morph true -model-file model_name.marmot subtag-separator "." -type-dict saldo_marmot.txt,indexes=[2,3] https://creativecommons.org/licenses/by/4.0/ University of Gothenburg Göteborgs universitet 2024-01-01T00:00:00Z swe