wehub-resource-sync
813 lines
86 KiB
Plaintext
813 lines
86 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5840e900-43cb-4ab4-81a5-988b68fda9b1",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 3.1 序列分类任务"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "958e7b5f-759a-431c-8af0-325271facb41",
|
||
"metadata": {},
|
||
"source": [
|
||
"基于 GPT-2 模型，可以通过微调（fine-tuning）或使用提示（prompt-based）方法来完成多种下游任务。\n",
|
||
"本章主要使用经典的微调方式，提示微调则属于chatgpt的范围，放在下一章，以下是几种常见的下游任务及其简单描述：\n",
|
||
"\n",
|
||
"\n",
|
||
"### 1. **文本分类**\n",
|
||
"\n",
|
||
"#### 任务描述\n",
|
||
"\n",
|
||
"文本分类是将文本分配到一个或多个预定义类别中的任务。例如，情感分析、主题分类等。生物序列中对应如启动序列等分类问题。\n",
|
||
"\n",
|
||
"#### 使用的模型类型\n",
|
||
"\n",
|
||
"- **GPT2ForSequenceClassification或AutoModelForSequenceClassification**：该模型在 GPT-2 的基础上添加了一个分类头，用于处理文本分类任务。通过微调这个模型，可以将其应用于多种分类任务。\n",
|
||
"\n",
|
||
"### 2. **机器翻译**\n",
|
||
"\n",
|
||
"#### 任务描述\n",
|
||
"\n",
|
||
"机器翻译是指将一种语言的文本转换为另一种语言的过程。生物学中，可以是生物序列到功能描述(英文)的翻译。\n",
|
||
"\n",
|
||
"#### 使用的模型类型\n",
|
||
"\n",
|
||
"- **AutoModelForSeq2SeqLM**：虽然 GPT-2 不是专门为机器翻译设计的模型，但可以通过构造特定格式的提示，让 GPT-2 根据上下文生成目标语言的翻译结果。\n",
|
||
"- **注意**：对于机器翻译任务，通常更推荐使用专门为此类任务设计的模型，如 T5 或 mBART。\n",
|
||
"\n",
|
||
"### 3. **词性标注 (POS Tagging)**\n",
|
||
"\n",
|
||
"#### 任务描述\n",
|
||
"\n",
|
||
"词性标注是指为每个单词分配其正确的词性标签（如名词、动词、形容词等）。生物学中，对应于结构预测任务，典型的如二级结构预测。\n",
|
||
"\n",
|
||
"#### 使用的模型类型\n",
|
||
"\n",
|
||
"- **AutoModelForTokenClassification**：该模型适用于标记级别的分类任务。通过微调，可以将 GPT-2 应用于词性标注，每个 token 的隐藏状态会被映射到相应的词性标签。\n",
|
||
"\n",
|
||
"### 4. **命名实体识别 (NER)**\n",
|
||
"\n",
|
||
"#### 任务描述\n",
|
||
"\n",
|
||
"命名实体识别是指识别文本中的人名、地名、组织机构等实体，并对其进行分类。生物学中，也对应于结构预测任务，典型的如膜结构预测。和词性标注类似。\n",
|
||
"\n",
|
||
"#### 使用的模型类型\n",
|
||
"\n",
|
||
"- **AutoModelForTokenClassification**：类似于词性标注，该模型可以用于 NER 任务，通过对每个 token 进行分类来识别和标注命名实体。\n",
|
||
"\n",
|
||
"### 5. **问答系统**\n",
|
||
"\n",
|
||
"#### 任务描述\n",
|
||
"\n",
|
||
"问答系统旨在根据给定的问题从文档或知识库中提取答案。目前一些最新的生物学大模型论文中，输入是包含生物序列的问题，回答则也是混合式的。一般是生物学领域的QA。\n",
|
||
"\n",
|
||
"#### 使用的模型类型\n",
|
||
"\n",
|
||
"- **AutoModelForQuestionAnswering**：该模型专门用于问答任务，能够理解问题并从上下文中提取答案。通过微调，它可以适应特定领域的问答需求。\n",
|
||
"\n",
|
||
"### 6. **文本生成**\n",
|
||
"\n",
|
||
"#### 任务描述\n",
|
||
"\n",
|
||
"文本生成是指根据给定的提示或前缀生成连贯的文本内容。生物学中，对应新的序列生成，如产生全新的蛋白质序列。\n",
|
||
"\n",
|
||
"#### 使用的模型类型\n",
|
||
"\n",
|
||
"- **GPT2LMHeadModel**：这是 GPT-2 的标准语言模型版本，擅长生成自然流畅的文本。它可以根据输入的提示生成后续文本，广泛应用于创作、对话系统等领域。\n",
|
||
"\n",
|
||
"### 6. **回归问题**\n",
|
||
"\n",
|
||
"#### 任务描述\n",
|
||
"\n",
|
||
"生物序列相关的回归问题，输入为序列，输出为一个float值。\n",
|
||
"\n",
|
||
"#### 使用的模型类型\n",
|
||
"\n",
|
||
"- huggingface没有特定的header，但一般回归问题，输出使用一个线性层即可，设定损失函数为均方误差（MSE）即可。最简单的，就是使用AutoModelForTokenClassification，类别数设置为1，输出的label为实测float值即可。\n",
|
||
"一个官方推荐的 [例子](https://github.com/huggingface/transformers/blob/7ae6f070044b0171a71f3269613bf02fd9fca6f2/src/transformers/models/bert/modeling_bert.py#L1564-L1575)\n",
|
||
"\n",
|
||
"### 小结\n",
|
||
"\n",
|
||
"GPT-2 可以通过微调或提示工程应用于多种下游任务。不同的任务需要使用特定类型的模型，这些模型基于 GPT-2 并添加了额外的组件或进行了调整，以更好地适应特定的任务需求\n",
|
||
"\n",
|
||
"<img src=\"img/gpt2-ft.png\" width=\"800px\" />"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "eca17933-7b8f-44de-8c59-ea7a1c8a3b33",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"\"\\nimport os\\n\\n# 设置环境变量, autodl专区 其他idc\\nos.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'\\n\\n# 打印环境变量以确认设置成功\\nprint(os.environ.get('HF_ENDPOINT'))\\n\""
|
||
]
|
||
},
|
||
"execution_count": 1,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"import subprocess\n",
|
||
"import os\n",
|
||
"# 设置环境变量, autodl一般区域\n",
|
||
"result = subprocess.run('bash -c \"source /etc/network_turbo && env | grep proxy\"', shell=True, capture_output=True, text=True)\n",
|
||
"output = result.stdout\n",
|
||
"for line in output.splitlines():\n",
|
||
" if '=' in line:\n",
|
||
" var, value = line.split('=', 1)\n",
|
||
" os.environ[var] = value\n",
|
||
"\n",
|
||
"\"\"\"\n",
|
||
"import os\n",
|
||
"\n",
|
||
"# 设置环境变量, autodl专区 其他idc\n",
|
||
"os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'\n",
|
||
"\n",
|
||
"# 打印环境变量以确认设置成功\n",
|
||
"print(os.environ.get('HF_ENDPOINT'))\n",
|
||
"\"\"\""
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "108d9c3c-ae4d-4110-a532-a40a6fe1f9df",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from transformers import AutoTokenizer, AutoModel\n",
|
||
"from tokenizers import Tokenizer\n",
|
||
"from transformers import GPT2LMHeadModel, AutoConfig,GPT2Tokenizer\n",
|
||
"from transformers import AutoModelForSequenceClassification\n",
|
||
"from transformers import DataCollatorWithPadding"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "bcdc9f7a-1ea5-4647-b87e-ac72ddf17818",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"application/vnd.jupyter.widget-view+json": {
|
||
"model_id": "c2e31c61549449e78a4e1fe0e884233f",
|
||
"version_major": 2,
|
||
"version_minor": 0
|
||
},
|
||
"text/plain": [
|
||
"tokenizer_config.json: 0%| | 0.00/580 [00:00<?, ?B/s]"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"data": {
|
||
"application/vnd.jupyter.widget-view+json": {
|
||
"model_id": "da2009ca96634f759f052a9a4ff7e41e",
|
||
"version_major": 2,
|
||
"version_minor": 0
|
||
},
|
||
"text/plain": [
|
||
"vocab.json: 0%| | 0.00/642k [00:00<?, ?B/s]"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"data": {
|
||
"application/vnd.jupyter.widget-view+json": {
|
||
"model_id": "b6b6ec58d8cb4878aa2e0786ff0bbcf4",
|
||
"version_major": 2,
|
||
"version_minor": 0
|
||
},
|
||
"text/plain": [
|
||
"merges.txt: 0%| | 0.00/323k [00:00<?, ?B/s]"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"data": {
|
||
"application/vnd.jupyter.widget-view+json": {
|
||
"model_id": "5dbb5171eb6242bdbded42c87ef46c27",
|
||
"version_major": 2,
|
||
"version_minor": 0
|
||
},
|
||
"text/plain": [
|
||
"special_tokens_map.json: 0%| | 0.00/473 [00:00<?, ?B/s]"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"#set tokenizer\n",
|
||
"tokenizer = GPT2Tokenizer.from_pretrained(\"dnagpt/dna_gpt2_v0\")\n",
|
||
"tokenizer.pad_token = tokenizer.eos_token"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "0e930ef5-865a-4528-84b5-ddae6d710a99",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Some weights of GPT2ForSequenceClassification were not initialized from the model checkpoint at dnagpt/dna_gpt2_v0 and are newly initialized: ['score.weight']\n",
|
||
"You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"GPT2ForSequenceClassification(\n",
|
||
" (transformer): GPT2Model(\n",
|
||
" (wte): Embedding(30000, 768)\n",
|
||
" (wpe): Embedding(1024, 768)\n",
|
||
" (drop): Dropout(p=0.1, inplace=False)\n",
|
||
" (h): ModuleList(\n",
|
||
" (0-11): 12 x GPT2Block(\n",
|
||
" (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
|
||
" (attn): GPT2SdpaAttention(\n",
|
||
" (c_attn): Conv1D(nf=2304, nx=768)\n",
|
||
" (c_proj): Conv1D(nf=768, nx=768)\n",
|
||
" (attn_dropout): Dropout(p=0.1, inplace=False)\n",
|
||
" (resid_dropout): Dropout(p=0.1, inplace=False)\n",
|
||
" )\n",
|
||
" (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
|
||
" (mlp): GPT2MLP(\n",
|
||
" (c_fc): Conv1D(nf=3072, nx=768)\n",
|
||
" (c_proj): Conv1D(nf=768, nx=3072)\n",
|
||
" (act): NewGELUActivation()\n",
|
||
" (dropout): Dropout(p=0.1, inplace=False)\n",
|
||
" )\n",
|
||
" )\n",
|
||
" )\n",
|
||
" (ln_f): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
|
||
" )\n",
|
||
" (score): Linear(in_features=768, out_features=2, bias=False)\n",
|
||
")"
|
||
]
|
||
},
|
||
"execution_count": 3,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"#set model\n",
|
||
"model = AutoModelForSequenceClassification.from_pretrained('dnagpt/dna_gpt2_v0', num_labels=2)\n",
|
||
"model.config.pad_token_id = model.config.eos_token_id\n",
|
||
"model"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "bd14794b-e507-4c1d-be47-0e0144835f18",
|
||
"metadata": {},
|
||
"source": [
|
||
"在生物学中，**启动子（promoter）** 是一段特定的DNA序列，它位于基因的上游（通常是5'端），并且是转录起始的关键调控元件。启动子的主要功能是为RNA聚合酶提供结合位点，并招募其他转录因子，以启动基因转录过程。以下是关于启动子的一些重要概念和特点：\n",
|
||
"\n",
|
||
"### 启动子的功能\n",
|
||
"\n",
|
||
"1. **转录起始**：\n",
|
||
" - 启动子是基因表达的第一步，它决定了何时、何地以及多频繁地进行转录。\n",
|
||
" \n",
|
||
"2. **调控基因表达**：\n",
|
||
" - 不同类型的启动子可以调节不同组织或细胞类型中的基因表达水平。例如，在某些细胞中高度活跃而在其他细胞中不活跃。\n",
|
||
"\n",
|
||
"3. **与转录因子和其他蛋白质相互作用**：\n",
|
||
" - 启动子区域通常包含多个顺式作用元件（cis-regulatory elements），这些元件可以与特定的转录因子或其他调控蛋白结合，进一步精细调整基因表达。\n",
|
||
" \n",
|
||
" \n",
|
||
"在生物学中，启动子（promoter）序列的二分类问题通常是指将DNA序列分为两类：**启动子序列**和**非启动子序列**。这种分类任务的目标是通过机器学习或生物信息学方法来预测给定的DNA序列是否具有启动子功能。\n",
|
||
"\n",
|
||
"### 二分类问题中的两个类别\n",
|
||
"\n",
|
||
"1. **启动子序列（Promoter Sequences）**：\n",
|
||
" - 这些序列包含能够指导转录起始的调控元件，通常是位于基因5'端上游区域的一段DNA。\n",
|
||
" - 启动子序列可能含有特定的保守基序（motifs），如TATA盒、CAAT盒等，这些基序对于RNA聚合酶及其辅助因子的结合至关重要。\n",
|
||
"\n",
|
||
"2. **非启动子序列（Non-Promoter Sequences）**：\n",
|
||
" - 这类序列指的是那些不具有启动子功能的DNA片段。它们可以来自基因内部（编码区或内含子）、基因间区域（intergenic regions）或其他调控元件（如增强子、沉默子等），但明确不是启动子。\n",
|
||
" - 非启动子序列不具备启动转录的能力，或者至少在自然条件下不会作为主要的转录起始点。\n",
|
||
"\n",
|
||
"### 启动子的研究意义\n",
|
||
"\n",
|
||
"理解启动子的工作机制对于揭示基因表达调控网络非常重要。这不仅有助于基础科学研究，而且对于医学应用也有着深远的影响，比如开发新的治疗策略来纠正异常的基因表达模式，或者利用合成生物学设计定制化的基因表达系统。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"id": "aee08f3f-6cda-4975-8cb9-9a7bfacb9eac",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"application/vnd.jupyter.widget-view+json": {
|
||
"model_id": "82d2ec71cf6648469040897d9174a55f",
|
||
"version_major": 2,
|
||
"version_minor": 0
|
||
},
|
||
"text/plain": [
|
||
"README.md: 0%| | 0.00/314 [00:00<?, ?B/s]"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"data": {
|
||
"application/vnd.jupyter.widget-view+json": {
|
||
"model_id": "40183e0714ea4155a2c0772fb7c72a00",
|
||
"version_major": 2,
|
||
"version_minor": 0
|
||
},
|
||
"text/plain": [
|
||
"train-00000-of-00001.parquet: 0%| | 0.00/8.66M [00:00<?, ?B/s]"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"data": {
|
||
"application/vnd.jupyter.widget-view+json": {
|
||
"model_id": "8e5ebe15df194e3c8bf5811777755947",
|
||
"version_major": 2,
|
||
"version_minor": 0
|
||
},
|
||
"text/plain": [
|
||
"Generating train split: 0%| | 0/59195 [00:00<?, ? examples/s]"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"from datasets import load_dataset\n",
|
||
"# 1. load ~11k samples from promoters prediction dataset\n",
|
||
"dataset = load_dataset(\"dnagpt/dna_promoter_300\")['train'].train_test_split(test_size=0.1)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"id": "6ac9fe5b-2175-42d8-949c-cb12bc8fb65c",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"DatasetDict({\n",
|
||
" train: Dataset({\n",
|
||
" features: ['sequence', 'label'],\n",
|
||
" num_rows: 53275\n",
|
||
" })\n",
|
||
" test: Dataset({\n",
|
||
" features: ['sequence', 'label'],\n",
|
||
" num_rows: 5920\n",
|
||
" })\n",
|
||
"})"
|
||
]
|
||
},
|
||
"execution_count": 9,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"dataset"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"id": "b5025f95-ca5d-42b1-95e1-55495f77d009",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"{'sequence': 'CCTGACGCCCACCGCAAGCTGCCGGGTAAGACCGGGTCGACTTCAGCGCGGCCCGCTGCACGAGAGACCATTATGGTGATCCGCCCGCCTGACACTACTGATATGTTGGGATTACAGGCGTGAGCCACGGCGCCCGGCGGGCAAGACACCCTCAGAGCACAGGGTGAATCCATGGTTAAAATACAGCGGGAAGTTAGCGCCGAAGTCGCCGTGTAATTTGTGCGCGGTTCAGGTTCATGTATTCAGAATCATTTTACTAGGTTTAGGGCTCGCCGCTGCCTCAGTGGCTTTCAGGCGCTT',\n",
|
||
" 'label': 0}"
|
||
]
|
||
},
|
||
"execution_count": 10,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"dataset[\"train\"][0]"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"id": "ac999213-67b1-4294-8d92-80b8c6c68acd",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"dna datasets mean token lenght 52.41266891891892 min token length 33 max token length 60\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"token_len_list = []\n",
|
||
"for item in dataset[\"test\"]:\n",
|
||
" inputs = tokenizer.tokenize(item[\"sequence\"])\n",
|
||
" token_len_list.append( len(inputs) )\n",
|
||
"\n",
|
||
"mean_len = sum(token_len_list)/len(token_len_list)\n",
|
||
"min_len = min(token_len_list)\n",
|
||
"max_len = max(token_len_list)\n",
|
||
"\n",
|
||
"print(\"dna datasets \", \"mean token lenght\", mean_len, \"min token length\", min_len, \"max token length\", max_len)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 14,
|
||
"id": "72a2dec3-043b-41e4-afd8-4dbd8c8fcbb0",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA1IAAAIjCAYAAAAJLyrXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy80BEi2AAAACXBIWXMAAA9hAAAPYQGoP6dpAABdnElEQVR4nO3deZyN9f//8eeZfTEzxjIbZowte2NLEyIUkQotiqylLBXSotVSREiopG9ZKh+lT4sP2UVCkrJlrNHImGEyjDH7zPX7Y36OTmO7zpwzZ4bH/XY7N3Nd1/v1vt7nOE7z7Hpf72MxDMMQAAAAAOCqubl6AAAAAABQ2hCkAAAAAMAkghQAAAAAmESQAgAAAACTCFIAAAAAYBJBCgAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACYRpADAyUaPHi2LxVIs52rTpo3atGlj3V63bp0sFou+/PLLYjl/3759VbVq1WI5l73S0tL06KOPKiwsTBaLRcOGDXPq+c7//ScnJzv1PNe6vn37qkyZMq4eBgBYEaQAwIS5c+fKYrFYHz4+PoqIiFCHDh00ffp0nT171iHnSUhI0OjRo7V9+3aH9OdIJXlsV2P8+PGaO3euBg0apE8++USPPPJIoTbnw8+VHv8MraVBcQdrs9LT0zV69GitW7fO1UMBgCvycPUAAKA0Gjt2rKKjo5WTk6PExEStW7dOw4YN09SpU7V48WI1bNjQ2vbll1/WCy+8YKr/hIQEjRkzRlWrVlVMTMxV161cudLUeexxubF9+OGHys/Pd/oYimLt2rW6+eab9dprr12yTbdu3VSjRg3rdlpamgYNGqSuXbuqW7du1v2hoaFOHev1Jj09XWPGjJGkUhdSAVx/CFIAYIc777xTTZs2tW6PGjVKa9eu1V133aW7775bcXFx8vX1lSR5eHjIw8O5H7fp6eny8/OTl5eXU89zJZ6eni49/9U4ceKE6tate9k2DRs2tAnDycnJGjRokBo2bKhevXo5e4gAgFKAqX0A4CBt27bVK6+8oj///FOffvqpdf/F7pFatWqVWrZsqbJly6pMmTK64YYb9OKLL0oqmH7VrFkzSVK/fv2s08jmzp0rqeD/1NevX1/btm3TrbfeKj8/P2vtv++ROi8vL08vvviiwsLC5O/vr7vvvltHjx61aVO1alX17du3UO0/+7zS2C52j9S5c+f0zDPPqEqVKvL29tYNN9ygyZMnyzAMm3YWi0VDhw7VN998o/r168vb21v16tXT8uXLL/6C/8uJEyc0YMAAhYaGysfHRzfeeKPmzZtnPX5+Wtvhw4e1dOlS69iPHDlyVf1fzNq1a9WqVSv5+/urbNmyuueeexQXF3fFuj///FM1atRQ/fr1lZSUJEk6ffq0hg0bZn2datSooYkTJ9pc4Tty5IgsFosmT56s2bNnq3r16vL29lazZs20detWu5/HvzljLIsWLVLdunXl4+Oj+vXr6+uvv7Z5vxw5ckQVK1aUJI0ZM8b69zN69Gibfo4dO6Z7771XZcqUUcWKFTVy5Ejl5eXZtFm4cKGaNGmigIAABQYGqkGDBnrnnXcc9voAgMQVKQBwqEceeUQvvviiVq5cqccee+yibX7//XfdddddatiwocaOHStvb28dPHhQGzdulCTVqVNHY8eO1auvvqqBAweqVatWkqRbbrnF2sfff/+tO++8Uz169FCvXr2uOMXsjTfekMVi0fPPP68TJ05o2rRpat++vbZv3269cnY1rmZs/2QYhu6++259//33GjBggGJiYrRixQo9++yzOnbsmN5++22b9j/++KO++uorDR48WAEBAZo+fbq6d++u+Ph4lS9f/pLjysjIUJs2bXTw4EENHTpU0dHRWrRokfr27avTp0/r6aefVp06dfTJJ59o+PDhqly5sp555hlJsv7ybtbq1at15513qlq1aho9erQyMjI0Y8YMtWjRQr/++uslF904dOiQ2rZtq3LlymnVqlWqUKGC0tPT1bp1ax07dkyPP/64IiMjtWnTJo0aNUrHjx/XtGnTbPpYsGCBzp49q8cff1wWi0WTJk1St27d9McffxT5qqAzxrJ06VI9+OCDatCggSZMmKCUlBQNGDBAlSpVsvZTsWJFvf/++4WmUP7zymBeXp46dOig5s2ba/LkyVq9erWmTJmi6tWra9CgQZIK/ifFQw89pHbt2mnixImSpLi4OG3cuFFPP/10kV4bALBhAACu2pw5cwxJxtatWy/ZJigoyGjUqJF1+7XXXjP++XH79ttvG5KMkydPXrKPrVu3GpKMOXPmFDrWunVrQ5Ixa9asix5r3bq1dfv77783JBmVKlUyUlNTrfu/+OILQ5LxzjvvWPdFRUUZffr0uWKflxtbnz59jKioKOv2N998Y0gyXn/9dZt29913n2GxWIyDBw9a90kyvLy8bPbt2LHDkGTMmDGj0Ln+adq0aYYk49NPP7Xuy87ONmJjY40yZcrYPPeoqCijc+fOl+3v306ePGlIMl577TXrvpiYGCMkJMT4+++/bcbr5uZm9O7d27rv/N//yZMnjbi4OCMiIsJo1qyZcerUKWubcePGGf7+/sb+/fttzvvCCy8Y7u7uRnx8vGEYhnH48GFDklG+fHmb+m+//daQZPzvf/+77PM4/35YtGjRJds4YywNGjQwKleubJw9e9a6b926dYYkm/fLxV7n8/r06WNIMsaOHWuzv1GjRkaTJk2s208//bQRGBho5ObmXva1AICiYmofADhYmTJlLrt6X9myZSVJ3377rd0LM3h7e6tfv35X3b53794KCAiwbt93330KDw/Xd999Z9f5r9Z3330nd3d3PfXUUzb7n3nmGRmGoWXLltnsb9++vapXr27dbtiwoQIDA/XHH39c8TxhYWF66KGHrPs8PT311FNPKS0tTevXr3fAs7ng+PHj2r59u/r27aty5crZjPf222+/6Ou6e/dutW7dWlWrVtXq1asVHBxsPbZo0SK1atVKwcHBSk5Otj7at2+vvLw8/fDDDzZ9Pfjggzb1568MXul1uhqOHktCQoJ27dql3r172yxf3rp1azVo0MD0+J544gmb7VatWtk877Jly+rcuXNatWqV6b4BwAyCFAA4WFpamk1o+bcHH3xQLVq00KOPPqrQ0FD16NFDX3zxhalQValSJVMLS9SsWdNm22KxqEaNGkW6P+hq/Pnnn4qIiCj0etSpU8d6/J8iIyML9REcHKyUlJQrnqdmzZpyc7P9z9qlzlNU5/u74YYbCh2rU6eOkpOTde7cOZv9Xbp0UUBAgFasWKHAwECbYwcOHNDy5ctVsWJFm0f79u0lFdz/9U//fp3OB5krvU5Xw9FjOf9a/XMVxPMutu9yfHx8Ck3F/Pf7Y/DgwapVq5buvPNOVa5cWf3797/q++wAwAzukQIAB/rrr7905syZy/6C6Ovrqx9++EHff/+9li5dquXLl+vzzz9X27ZttXLlSrm7u1/xPGbua7pal/rS4Ly8vKsakyNc6jzGvxamKI26d++uefPm6bPPPtPjjz9ucyw/P1+33367nnvuuYvW1qpVy2bbma9TSRrLv13N+zAkJETbt2/XihUrtGzZMi1btkxz5sxR7969bRYfAYCiIkgBgAN98sknkqQOHTpctp2bm5vatWundu3aaerUqRo/frxeeuklff/992rfvv0lQ429Dhw4YLNtGIYOHjxocyN/cHCwTp8+Xaj2zz//VLVq1azbZsYWFRWl1atX6+zZszZXpfbu3Ws97ghRUVHauXOn8vPzba5KOfo8/zyfJO3bt6/Qsb1796pChQry9/e32f/WW2/Jw8PDupDGww8/bD1WvXp1paWlWa/6uJKjx3L+tTp48GChY//e56j3vZeXl7p06aIuXbooPz9fgwcP1gcffKBXXnnF9FUwALgUpvYBgIOsXbtW48aNU3R0tHr27HnJdqdOnSq07/wX22ZlZUmS9ZfwiwUbe8yfP9/mvq0vv/xSx48f15133mndV716df3000/Kzs627luyZEmhZdLNjK1Tp07Ky8vTzJkzbfa//fbbslgsNucvik6dOikxMVGff/65dV9ubq5mzJihMmXKqHXr1g45z3nh4eGKiYnRvHnzbF6H3bt3a+XKlerUqVOhGovFotmzZ+u+++5Tnz59tHjxYuuxBx54QJs3b9aKFSsK1Z0+fVq5ubkOHf/lOHosERERql+/vubPn6+0tDTr/vXr12vXrl02bf38/Kznsdfff/9ts+3m5mb9Hwbn/30BgCNwRQoA7LBs2TLt3btXubm5SkpK0tq1a7Vq1SpFRUVp8eLF8vHxuWTt2LFj9cMPP6hz586KiorSiRMn9N5776ly5cpq2bKlpIJQU7ZsWc2aNUsBAQHy9/dX8+bNFR0dbdd4y5Urp5YtW6pfv35KSkrStGnTVKNGDZsl2h999FF9+eWX6tixox544AEdOnRIn376qc3iD2bH1qVLF91222166aWXdOTIEd14441auXKlvv32Ww0bNqxQ3/YaOHCgPvjgA/Xt21fbtm1T1apV9eWXX2rjxo2aNm3aZe9Zs9dbb72lO++8U7GxsRowYIB1+fOgoKBC3310npubmz799FPde++9euCBB/Tdd9+pbdu2evbZZ7V48WLddddd6tu3r5o0aaJz585p165d+vLLL3XkyBFVqFDBYWP/73//a71a9099+vRxyljGjx+ve+65Ry1atFC/fv2UkpKimTNnqn79+jbhytfXV3Xr1tXnn3+uWrVqqVy5cqpfv77q169/1ed69NFHderUKbVt21aVK1fWn3/+qRkzZigmJsZ6zxwAOIRL1wwEgFLm/PLn5x9eXl5GWFiYcfvttxvvvPOOzTLb5/17+fM1a9YY99xzjxEREWF4eXkZERERxkMPPVRouelvv/3WqFu3ruHh4WGz3Hjr1q2NevXqXXR8l1r+/D//+Y8xatQoIyQkxPD19TU6d+5s/Pnnn4Xqp0yZYlSqVMnw9vY2WrRoYfzyyy+F+rzc2P69/LlhGMbZs2eN4cOHGxEREYanp6dRs2ZN46233jLy8/Nt2kkyhgwZUmhMl1qW/d+SkpKMfv36GRUqVDC8vLyMBg0aXHSJdkctf24YhrF69WqjRYsWhq+vrxEYGGh06dLF2LNnj02bfy5/fl56errRunVro0yZMsZPP/1kGEbB6zRq1CijRo0ahpeXl1GhQgXjlltuMSZPnmxkZ2cbhnFhyfG33nqr0BgvNr5/O/9+uNRjw4YNThvLwoULjdq1axve3t5G/fr1jcWLFxvdu3c3ateubdNu06ZNRpMmTQwvLy+bfvr06WP4+/sXOte//319+eWXxh133GGEhIQYXl5eRmRkpPH4448bx48fv+xrAwBmWQzjGriDFwAAlDoxMTGqWLEiS5UDKJW4RwoAADhVTk5OoXur1q1bpx07dqhNmzauGRQAFBFXpAAAgFMdOXJE7du3V69evRQREaG9e/dq1qxZCgoK0u7du1W+fHlXDxEATGOxCQAA4FTBwcFq0qSJ/u///k8nT56Uv7+/OnfurDfffJMQBaDU4ooUAAAAAJjEPVIAAAAAYBJBCgAAAABM4h4pSfn5+UpISFBAQIAsFourhwMAAADARQzD0NmzZxURESE3t0tfdyJISUpISFCVKlVcPQwAAAAAJcTRo0dVuXLlSx4nSEkKCAiQVPBiBQYGung0AADgmlW7tnT8uBQeLu3da758Zm0dP3tc4QHh2jvUfD2AK0tNTVWVKlWsGeFSCFKSdTpfYGAgQQoAADjP+WlCbm6SHb9zuPm4STkFf/I7C+BcV7rlh8UmAAAAAMAkghQAAAAAmESQAgAAAACTuEcKAACguGzdKuXlSe7u9pU/tlV5Rp7cLfbVA3AcghQAAEBxCQ8vWnlA0eoBOA5T+wAAAADAJIIUAAAAAJjE1D4AAIDiMnu2lJYmlSkjDRxovnzbbKVlp6mMVxkNbGK+HoDjWAzDMFw9CFdLTU1VUFCQzpw5w5fbAQAA56lcWTp2TKpUSfrrL/PlUyvr2NljqhRQSX+NMF8P4MquNhswtQ8AAAAATCJIAQAAAIBJBCkAAAAAMIkgBQAAAAAmEaQAAAAAwCSCFAAAAACYRJACAAAAAJMIUgAAAABgkoerBwAAAHDdqFVLCgqSQkPtKy9fS0E+QQr1t68egONYDMMwXD0IV7vaby8GAAAoTvHx8UpOTnZK3xUqVFBkZKRT+gZKs6vNBlyRAgAAKIHi4+NVu04dZaSnO6V/Xz8/7Y2LI0wBdiJIAQAAlEDJycnKSE/X4MmzFVG9lkP7Tji0X++NHKjk5GSCFGAnghQAAEAJFlG9lqLrxbh6GAD+hSAFAABQXHr2lJKTpQoVpM8+M13+7sbHdDbrbwV4l9eQFh86YYAArhZBCgAAoLisXy8dOyZVqmRXedyJjUrJSFCwb4SDBwbALL5HCgAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAkwhSAAAAAGASX8gLAABQXB57TDpzRgoKsqv8thq9lZGdKl+vQAcPDIBZBCkAAIDi8tprRSrv3uAFBw0EQFExtQ8AAABF17evZLFITzxR+NiQIQXH+vYt7lFdnfNj/+ejY8cLx48ckQYMkKKjJV9fqXr1glCcnX11/RuGdOedBf1+843tsaeekpo0kby9pZgYxzwfFAuuSAEAAMAxqlSRFi6U3n67IHBIUmamtGCBFBnp2rFdSceO0pw5F7a9vS/8vHevlJ8vffCBVKOGtHt3wTTNc+ekyZOv3Pe0aQUh6lL695e2bJF27rR7+Ch+XJECAACAYzRuXBCmvvrqwr6vvioIUY0a2bbNz5cmTLhwlefGG6Uvv7xwPC/P9irQDTdI77xj20ffvtK99xaEmfBwqXz5gqtfOTnmx+7tLYWFXXgEB184dj5k3XGHVK2adPfd0siRts/zUrZvl6ZMkT7++OLHp08vGHO1aubHDJciSAEAABSXypULrkxUrmxX+dCv66rngrIa+nVdBw/Mgfr3t72y8/HHUr9+hdtNmCDNny/NmiX9/rs0fLjUq5e0fn3B8fz8gtdp0SJpzx7p1VelF1+UvvjCtp/vv5cOHSr4c948ae7cgsd5o0dLVateedzr1kkhIQWBbdAg6e+/L9/+zBmpXLnLt0lPlx5+WHr33YJwhmsKU/sAAADgOL16SaNGSX/+WbC9cWPBdL916y60ycqSxo+XVq+WYmML9lWrJv34Y8H0udatJU9PacyYCzXR0dLmzQVB6oEHLuwPDpZmzpTc3aXataXOnaU1awqm3klShQoF9zRdTseOUrduBec4dKggsN15Z8H53N0Ltz94UJox48rT+oYPl265Rbrnnsu3Q6lEkAIAAIDjVKxYEGbmzi1YZKFz54Iw808HDxZcrbn9dtv92dm2UwDffbfgilZ8vJSRUXD83wsy1KtnG3bCw6Vduy5sDx1a8LicHj0u/NyggdSwYUH4WrdOatfOtu2xYwXB6/77L4S1i1m8WFq7Vvrtt8ufG6UWQQoAAACO1b//hfDy7ruFj6elFfy5dKlUqZLtsfOLPCxcWHAf0pQpBVetAgKkt94qWJThnzw9bbctloJpgUVRrVpB+Dt40DZIJSRIt91WcJVp9uzL97F2bcHVrbJlbfd37y61amV7hQ6lEkEKAAAAjtWxY8HVI4tF6tCh8PG6dQsCU3x8wTS+i9m4sSCwDB58Yd+hQ84Z77/99VfBPVLh4Rf2HTtWEKKaNCm4B8ztCksNvPCC9OijtvsaNChY0bBLF8ePGcWOIAUAAADHcneX4uIu/PxvAQEFV5uGDy+4etSyZcHiDRs3SoGBUp8+Us2aBYtRrFhRcO/SJ59IW7cW/GzGzJnS118X3Dd1MWlpBfdide9esCDEoUPSc88VLHN+PgQeOya1aSNFRRXcF3Xy5IX684tIHDtWcPVq/nzpppsurP73b5GRts/h4MGCMSQmFkxf3L69YH/dupKXl7nnimJFkAIAAIDjBQZe/vi4cQX3U02YIP3xR8EUuMaNCxZ6kKTHHy+4v+jBBwuubD30UMHVqWXLzI0jOfnyV7Lc3Qu+v2nePOn0aSkiomCZ83HjLkwzXLWqIPAcPFh4xUXDKPgzJ0fat6/g3i8zHn30wkqF0oV7xA4fvrrVBuEyFsM4/7d//UpNTVVQUJDOnDmjwCv9owcAALBX5coFVy4qVSqYPnYZv/76q5o0aaLXv16n6HoxkgqWP0/JSFCwb4Rmdt1j9zAO/75dL3dto23btqlx48Z29wNci642G/A9UgAAAABgEkEKAAAAAEziHikAAIDi8umnBV9Ge/7eG5MG3/KBcvOy5eHOIgSAqxGkAAAAikubNkUqrxvayjHj+P/izq+s5wQVKlRQZGSk0/oHXI0gBQAAcJ05fTJJFotFvXr1cto5fP38tDcujjCFaxZBCgAA4DqTnnpGhmGo37jpql6/ocP7Tzi0X++NHKjk5GSCFK5ZBCkAAIDism7dhXuk7Jjmtydpg/UeKUdM8wuPrmFdWh2AOQQpAACA4tKr11V/j9TFvLfpcYd8jxSAomP5cwAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAkwhSAAAAAGASQQoAAAAATOJ7pAAAAOwUHx+v5OTkq25fPydHXpKyc3K0+9dfL9s2Li6uiKMD4EwEKQAAADvEx8erdp06ykhPv+qao5IqSzpx4oSaNGlyVTXZWdn2DRCAUxGkAAAA7JCcnKyM9HQNnjxbEdVrXVXNvH/8/PoV2u5Yv0qLpr2h3Nxc676ZXfeYHygApyBIAQAAFEFE9VqKrhfj8H4TDu13eJ8AHIfFJgAAAADAJIIUAAAAAJjE1D4AAIBiUn/Gm/I6m6rsgEDtfvIF0/X/3fWmMrJT5esVqO4NzNcDcByCFAAAQDGp8cV8+SUlKD00wq4g9f3B+UrJSFCwbwRBCnAxpvYBAAAAgEkEKQAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAk1wapPLy8vTKK68oOjpavr6+ql69usaNGyfDMKxtDMPQq6++qvDwcPn6+qp9+/Y6cOCATT+nTp1Sz549FRgYqLJly2rAgAFKS0sr7qcDAAAA4Drh0iA1ceJEvf/++5o5c6bi4uI0ceJETZo0STNmzLC2mTRpkqZPn65Zs2Zpy5Yt8vf3V4cOHZSZmWlt07NnT/3+++9atWqVlixZoh9++EEDBw50xVMCAAAAcB1w6Rfybtq0Sffcc486d+4sSapatar+85//6Oeff5ZUcDVq2rRpevnll3XPPfdIkubPn6/Q0FB988036tGjh+Li4rR8+XJt3bpVTZs2lSTNmDFDnTp10uTJkxUREeGaJwcAAPAvJ25qIe+Uv5UVXN6u+johLXQ2628FeNtXD8BxXBqkbrnlFs2ePVv79+9XrVq1tGPHDv3444+aOnWqJOnw4cNKTExU+/btrTVBQUFq3ry5Nm/erB49emjz5s0qW7asNURJUvv27eXm5qYtW7aoa9euhc6blZWlrKws63ZqaqoTnyUAAECBTVM+LFL9kBZFqwfgOC4NUi+88IJSU1NVu3Ztubu7Ky8vT2+88YZ69uwpSUpMTJQkhYaG2tSFhoZajyUmJiokJMTmuIeHh8qVK2dt828TJkzQmDFjHP10AAAAAFwnXHqP1BdffKHPPvtMCxYs0K+//qp58+Zp8uTJmjdvnlPPO2rUKJ05c8b6OHr0qFPPBwClwboj62QZY9HpzNOSpLnb56rsm2VdOiYAAEoqlwapZ599Vi+88IJ69OihBg0a6JFHHtHw4cM1YcIESVJYWJgkKSkpyaYuKSnJeiwsLEwnTpywOZ6bm6tTp05Z2/ybt7e3AgMDbR4AUJL1/aavLGMsemLJE4WODVk6RJYxFvX9pq9Dz/lgvQe1/8n9Du3zalWdVlWWMRabx5s/vmk9vi95n26bd5tCJ4fK53UfVXunml5e+7Jy8nIu22/8mXh1XtBZfm/4KeStED278lnl5ufatFl3ZJ0af9BY3q97q8b0Gpq7fa4zniIAoJRz6dS+9PR0ubnZZjl3d3fl5+dLkqKjoxUWFqY1a9YoJiZGUsH9TFu2bNGgQYMkSbGxsTp9+rS2bdumJk2aSJLWrl2r/Px8NW/evPieDAA4WZXAKlq4e6He7vC2fD19JUmZuZlasHuBIoMiHX4+X09f63lcYWybsXqsyWPW7QCvAOvPnu6e6t2wtxqHN1ZZn7LakbRDj/3vMeUb+RrfbvxF+8vLz1PnBZ0VViZMmwZs0vGzx9X7m97ydPe01hxOOazOCzrriSZP6LNun2nN4TV6dPGjCi8Trg41Ojj3CeO60LZ3F/kkn1RmhYpaO/9/puvfWNNFZzJPKsinol5qZ74egOO4NEh16dJFb7zxhiIjI1WvXj399ttvmjp1qvr37y9JslgsGjZsmF5//XXVrFlT0dHReuWVVxQREaF7771XklSnTh117NhRjz32mGbNmqWcnBwNHTpUPXr0YMU+ANeUxuGNdSjlkL6K+0o9GxbcS/pV3FeKDIpUdNlom7b5Rr4m/jhRs3+drcS0RNUqX0uv3PqK7qt7n7XNdwe+07Dlw3Q09ahurnyz+tzYx6aPudvnatjyYTr9wmlJ0qFThzRi5Qj99NdPOpd9TnUq1tGEdhPUvtqFBYGqTquqgU0G6uCpg1q0Z5GCfYL18q0va2AT819JEeAdoLAyF59ZUC24mqoFV7NuR5WN0roj67QhfsMl+1t5aKX2nNyj1Y+sVmiZUMWExWjcbeP0/OrnNbrNaHm5e2nWL7MUXTZaUzpMkSTVqVhHP8b/qLd/epsgBYcIPHxIfkkJSj9r30JXx1MPKSUjQenZLJQFuJpLp/bNmDFD9913nwYPHqw6depo5MiRevzxxzVu3Dhrm+eee05PPvmkBg4cqGbNmiktLU3Lly+Xj4+Ptc1nn32m2rVrq127durUqZNatmyp2bNnu+IpAYBT9Y/prznb51i3P/7tY/WL6Veo3YQNEzR/53zN6jxLvw/+XcNvHq5eX/XS+iPrJUlHzxxVt8+7qUutLtr++HY92uhRvbD6hcueOy07TZ1qdNKa3mv02+O/qWP1juryny6KPxNv027K5ilqGtFUvz3+mwY3G6xBSwdpX/I+6/E2c9tc1TTEN398U+UnlVejDxrprY1vFZqC908HTx3U8oPL1Tqq9SXbbP5rsxqENFBomQsLGHWo3kGpWan6/cTv1jb/DIbn22z+a/MVxwsAuL649IpUQECApk2bpmnTpl2yjcVi0dixYzV27NhLtilXrpwWLFjghBECQMnSq2EvjVozSn+e/lOStPHoRi28b6HWHVlnbZOVm6XxP47X6kdWK7ZKrKSCKzg/xv+oD7Z9oNZVW+v9X95X9XLVrVdebqhwg3ad2KWJGyde8tw3ht2oG8NutG6PaztOX+/9Wov3LdbQm4Za93eq2UmDmw2WJD3f4nm9/dPb+v7I97qhwg2SpMigSIWXCb/s83yq+VNqHN5Y5XzLadPRTRq1ZpSOpx3X1A5Tbdrd8tEt+vX4r8rKy9LAxgM19rZL/7ciMS3RJkRJsm4npiVeaONfuE1qVqoycjJcOtURAFCyuDRIAQDMqehfUZ1rddbc7XNlyFDnmp1Vwa+CTZuDpw4qPSddt39yu83+7LxsNQpvJEmKS45T80q295HGVo697LnTstM0et1oLT2wVMfPHldufq4ycjMKXZFqGNLQ+rPFYlFYmTCdOHdhUaD5Xedf8XmOiB1xob/QhvJy99LjSx7XhHYT5O3hbT32+X2f62z2We1I3KFnVz2ryZsm67kWz12xfwAAioogBQClTP+Y/hq6rOAK0Lud3i10PC07TZK09OGlqhRYyeaYt7t3ofZXa+TKkVr1xypNvn2yapSrIV9PX933xX3Kzsu2aefp7mmzbZFF+Ua+3eeVpOaVmis3P1dHTh+xXtmSpCpBVSRJdSvWVZ6Rp4H/G6hnYp+Ru5t7oT7CyoTp52M/2+xLSkuyHjv/Z9K5pEJtAr0DuRoFALBBkAKAUqZjjY7KzsuWRRZ1qF54AYS6FevK291b8Wfi1brqxe8ZqlOhjhbvW2yz76e/frrseTce3ai+N/ZV1zpdJRUEtiOnj9j3JEzanrhdbhY3hfiHXLJNvpGvnPwc5Rv5clfhIBVbOVZvbHhDJ86dsPaz6o9VCvQOVN2Kda1tvjv4nU3dqj9WXfFqHQDg+kOQAoBSxt3NXXFD4qw//1uAd4BG3jJSw1cMV76Rr5aRLXUm64w2xm9UoHeg+sT00RNNn9CUzVP07Mpn9WjjR7Xt+DbN3TH3suetWa6mvtr7lbrc0EUWWfTK96/YdaWp99e9VSmgkia0n3DR45uPbtaWY1t0W9XbFOAdoM1HN2v4iuHq1bCXgn2DJUmf7fxMnu6eahDSQN4e3vol4ReNWjNKD9Z70HpF7Ou4rzVqzSjtHbpXknRH9TtUt2JdPfL1I5rUfpIS0xL18tqXNaTZEOt0wSeaPqGZW2fquVXPqX+j/lp7eK2++P0LLX14qennCQC4thGkAKAUCvS+/BeJj7ttnCr6VdSEHyfoj5Q/VNanrBqHN9aLrV6UVLDgw38f+K+GrxiuGT/P0E2VbtL4tuPVf3H/S/Y5tcNU9f+2v2756BZV8Kug51s8r9Qs80swx5+Jl5vl0ovGent4a+HuhRq9brSy8rIUXTZaw28ebnPflIebhyZunKj9f++XYRiKKhuloc2GanjscGubM1lntO/vC6sFuru5a8lDSzRo6SDFfhQrfy9/9bmxj80CFdHB0Vr68FINXzFc72x5R5UDK+v/7v4/lj4HABRiMQzDcPUgXC01NVVBQUE6c+aMAgMv/8sJAACAJP36669q0qSJXv96naLrxVxVzb2t6hZ8j1RohL7ZsOeybTcu/kLvjRyoFz/9TvVuukWSNPTrukrJSFCwb4Rmdr18vdm+Henw79v1ctc22rZtmxo3buzw/gFnutpswBUpAACAYrJryHPyTE9Tjl8Zu+q7NXhOmTlp8vG0rx6A4xCkAAAAismhHn2LVN+2RtHqATjOpSepAwAAAAAuiiAFAAAAACYxtQ8AAKCY+JxIlCU/T4abuzJDwkzXp2QkKt/Ik5vFXcG+5usBOA5BCgAAoJh07N72qlftu5hXlrd1yKp9AIqOqX0AAAAAYBJBCgAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAkwhSAAAAAGCSh6sHAAAAcL1YM+9bueXlKt/dvl/BXmz3rfLzc+Xmxq9wgKvxrxAAAKCYnK1Ws0j1EYFFqwfgOEztAwAAAACTCFIAAAAAYBJT+wAAAIpJ1P8WySMjQ7m+vvqzy/2m6zceWaTs3Ax5efiqRVXz9QAchyAFAABQTBpNek1+SQlKD42wK0j957fXlJKRoGDfCIIU4GJM7QMAAAAAkwhSAAAAAGASQQoAAAAATCJIAQAAAIBJBCkAAAAAMIkgBQAAAAAmEaQAAAAAwCSCFAAAAACYxBfyAgAAFJOMiiE2f5pV1jfE5k8ArkOQAgAAKCYrvlpXpPrXOxatHoDjEKQAAMA1LT4+XsnJyQ7vNy4uzuF9Aig9CFIAAOCaFR8fr9p16igjPd1p58jOynZa3wBKLoIUAAC4ZiUnJysjPV2DJ89WRPVaDu17x/pVWjTtDeXm5jq0XwClA0EKAABc8yKq11J0vRiH9plwaL/pmmavDJP3mRRlBQVr67hppus/+nmY0rJSVMY7WANuMl8PwHEIUgAAAMWk0rqV8ktKUHpohLbaUf/bsZVKyUhQsG+Ew8cGwBy+RwoAAAAATCJIAQAAAIBJBCkAAAAAMIkgBQAAAAAmEaQAAAAAwCSCFAAAAACYRJACAAAAAJMIUgAAAABgEl/ICwAAUEyO3NVdXmdOKzuorF31t1TtrnPZp+XvZV89AMchSAEAABST7c+PK1L9w42KVg/AcZjaBwAAAAAmEaQAAAAAwCSCFAAAAACYxD1SAAAAxaRzh2byO5Go9JAwLV2x1XT9yCXNlJKeqGC/ME2+y3w9AMfhihQAAEAx8Uw/J89zZ+WZfs6u+sycc8rMPavMHPvqATgOQQoAAAAATCJIAQAAAIBJBCkAAAAAMIkgBQAAAAAmEaQAAAAAwCSCFAAAAACYRJACAAAAAJMIUgAAAABgkoerBwAAAHC9+HnsVLlnZirPx8eu+v43TVVOXqY83e2rB+A4BCkAAIBiknBbxyLVN65UtHoAjsPUPgAAAAAwiSAFAAAAACYxtQ8AAKCYBO/eLvecbOV5eimlfozp+sOntis3L1se7l6KLme+HoDjEKQAAACKSetBD8svKUHpoRH6ZsMe0/VT1j+slIwEBftGaGZX8/UAHIcgBQAAAKeIi4tzSr8VKlRQZGSkU/oGrhZBCgAAAA51+mSSLBaLevXq5ZT+ff38tDcujjAFlyJIAQAAwKHSU8/IMAz1Gzdd1es3dGjfCYf2672RA5WcnEyQgksRpAAAAOAU4dE1FF0vxtXDAJyC5c8BAAAAwCSCFAAAAACYRJACAAAAAJMIUgAAAABgEkEKAAAAAExi1T4AAIBismTZFkmGJItd9W/dtUWGDFnsrAfgOAQpAACAYpJbJqBI9b6eRasH4DhM7QMAAAAAkwhSAAAAAGCSy4PUsWPH1KtXL5UvX16+vr5q0KCBfvnlF+txwzD06quvKjw8XL6+vmrfvr0OHDhg08epU6fUs2dPBQYGqmzZshowYIDS0tKK+6kAAABcVu2PZ6rB9Amq/fFMu+q/i5up/+6coO/i7KsH4DguDVIpKSlq0aKFPD09tWzZMu3Zs0dTpkxRcHCwtc2kSZM0ffp0zZo1S1u2bJG/v786dOigzMxMa5uePXvq999/16pVq7RkyRL98MMPGjhwoCueEgAAwCXVnvOeGsycqNpz3rOr/ru97+mr3RP13V776gE4jksXm5g4caKqVKmiOXPmWPdFR0dbfzYMQ9OmTdPLL7+se+65R5I0f/58hYaG6ptvvlGPHj0UFxen5cuXa+vWrWratKkkacaMGerUqZMmT56siIiI4n1SAAAAAK55Lr0itXjxYjVt2lT333+/QkJC1KhRI3344YfW44cPH1ZiYqLat29v3RcUFKTmzZtr8+bNkqTNmzerbNmy1hAlSe3bt5ebm5u2bNly0fNmZWUpNTXV5gEAAAAAV8ulQeqPP/7Q+++/r5o1a2rFihUaNGiQnnrqKc2bN0+SlJiYKEkKDQ21qQsNDbUeS0xMVEhIiM1xDw8PlStXztrm3yZMmKCgoCDro0qVKo5+agAAAACuYS4NUvn5+WrcuLHGjx+vRo0aaeDAgXrsscc0a9Ysp5531KhROnPmjPVx9OhRp54PAAAAwLXFpUEqPDxcdevWtdlXp04dxcfHS5LCwsIkSUlJSTZtkpKSrMfCwsJ04sQJm+O5ubk6deqUtc2/eXt7KzAw0OYBAAAAAFfLpUGqRYsW2rdvn82+/fv3KyoqSlLBwhNhYWFas2aN9Xhqaqq2bNmi2NhYSVJsbKxOnz6tbdu2WdusXbtW+fn5at68eTE8CwAAAADXG5eu2jd8+HDdcsstGj9+vB544AH9/PPPmj17tmbPni1JslgsGjZsmF5//XXVrFlT0dHReuWVVxQREaF7771XUsEVrI4dO1qnBObk5Gjo0KHq0aMHK/YBAAAAcAqXBqlmzZrp66+/1qhRozR27FhFR0dr2rRp6tmzp7XNc889p3PnzmngwIE6ffq0WrZsqeXLl8vHx8fa5rPPPtPQoUPVrl07ubm5qXv37po+fbornhIAAACA64BLg5Qk3XXXXbrrrrsuedxisWjs2LEaO3bsJduUK1dOCxYscMbwAAAAHOZUvYZKD6+kzHLl7aqPLtdQ5TMrKdDHvnoAjuPyIAUAAHC9+GHWwiLVP9O6aPUAHMeli00AAAAAQGlEkAIAAAAAkwhSAAAAAGAS90gBAAAUk1uf6CGfU38rs1x5u+6XmrK+h1Iz/1agT3nulwJcjCAFAABQTMr9vlN+SQlKD7Xvuy4Pn9qplIwEBfvyXZmAqzG1DwAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAkwhSAAAAAGASX8gLAABQTPb2GyzPtLPKKRNgV32n2oOVkXNWvp721QNwHLuC1B9//KFq1ao5eiwAAADXtL39hxapvlOdotUDcBy7pvbVqFFDt912mz799FNlZmY6ekwAAAAAUKLZFaR+/fVXNWzYUCNGjFBYWJgef/xx/fzzz44eGwAAAACUSHYFqZiYGL3zzjtKSEjQxx9/rOPHj6tly5aqX7++pk6dqpMnTzp6nAAAAKWeR9pZeaSlyiPtrF31GTlnlZ6Tqowc++oBOE6RVu3z8PBQt27dtGjRIk2cOFEHDx7UyJEjVaVKFfXu3VvHjx931DgBAABKvbvubK4HGkfqrjub21X/7JLmemxRpJ5dYl89AMcpUpD65ZdfNHjwYIWHh2vq1KkaOXKkDh06pFWrVikhIUH33HOPo8YJAAAAACWGXav2TZ06VXPmzNG+ffvUqVMnzZ8/X506dZKbW0Eui46O1ty5c1W1alVHjhUAAAAASgS7gtT777+v/v37q2/fvgoPD79om5CQEH300UdFGhwAAAAAlER2BakDBw5csY2Xl5f69OljT/cAAAAAUKLZdY/UnDlztGjRokL7Fy1apHnz5hV5UAAAAABQktkVpCZMmKAKFSoU2h8SEqLx48cXeVAAAAAAUJLZFaTi4+MVHR1daH9UVJTi4+OLPCgAAAAAKMnsClIhISHauXNnof07duxQ+fLlizwoAAAAACjJ7Fps4qGHHtJTTz2lgIAA3XrrrZKk9evX6+mnn1aPHj0cOkAAAOB68fHxSk5OdkrfFSpUUGRkpFP6BgBnsStIjRs3TkeOHFG7du3k4VHQRX5+vnr37s09UgAAXGPi4+NVu04dZaSnO6V/Xz8/7Y2Luy7C1Pr3F8g9J1t5nl521T/TeoFy87Ll4W5fPQDHsStIeXl56fPPP9e4ceO0Y8cO+fr6qkGDBoqKinL0+AAAgIslJycrIz1dgyfPVkT1Wg7tO+HQfr03cqCSk5OviyCVUj+mSPXR5YpWD8Bx7ApS59WqVUu1ajn2AxUAAJRMEdVrKbpejKuHAQAlgl1BKi8vT3PnztWaNWt04sQJ5efn2xxfu3atQwYHAAAAACWRXUHq6aef1ty5c9W5c2fVr19fFovF0eMCAAC45kR8v1zumZnK8/FRwm0dTdf/emy5cvIy5enuo8aVzNcDcBy7gtTChQv1xRdfqFOnTo4eDwAAwDXrpldHyC8pQemhEfpmg/kg9PHPI5SSkaBg3wg17kqQAlzJru+R8vLyUo0aNRw9FgAAAAAoFewKUs8884zeeecdGYbh6PEAAAAAQIln19S+H3/8Ud9//72WLVumevXqydPT0+b4V1995ZDBAQAAAEBJZFeQKlu2rLp27erosQAAAABAqWBXkJozZ46jxwEAAAAApYZd90hJUm5urlavXq0PPvhAZ8+elSQlJCQoLS3NYYMDAAAAgJLIritSf/75pzp27Kj4+HhlZWXp9ttvV0BAgCZOnKisrCzNmjXL0eMEAAAAgBLDritSTz/9tJo2baqUlBT5+vpa93ft2lVr1qxx2OAAAAAAoCSy64rUhg0btGnTJnl5ednsr1q1qo4dO+aQgQEAAFxrcvz8leMfoBw/f7vqfTz95ZMTIB9P++oBOI5dQSo/P195eXmF9v/1118KCAgo8qAAAACuRUtXbC1S/eS7ilYPwHHsmtp3xx13aNq0adZti8WitLQ0vfbaa+rUqZOjxgYAAAAAJZJdV6SmTJmiDh06qG7dusrMzNTDDz+sAwcOqEKFCvrPf/7j6DECAAAAQIliV5CqXLmyduzYoYULF2rnzp1KS0vTgAED1LNnT5vFJwAAAADgWmRXkJIkDw8P9erVy5FjAQAAuKbFTHxFXmdOKzuorLY/P850/YLfXtG57NPy9yqrhxuZrwfgOHYFqfnz51/2eO/eve0aDAAAwLWs6pL/yi8pQemhEXYFqU1H/quUjAQF+0YQpAAXsytIPf300zbbOTk5Sk9Pl5eXl/z8/AhSAAAAAK5pdq3al5KSYvNIS0vTvn371LJlSxabAAAAAHDNsytIXUzNmjX15ptvFrpaBQAAAADXGocFKalgAYqEhARHdgkAAAAAJY5d90gtXrzYZtswDB0/flwzZ85UixYtHDIwAAAAACip7ApS9957r822xWJRxYoV1bZtW02ZMsUR4wIAAACAEsuuIJWfn+/ocQAAAABAqeHQe6QAAAAA4Hpg1xWpESNGXHXbqVOn2nMKAACAa86xNnfI+0yKsoKC7apvVOkOpWWlqIy3ffUAHMeuIPXbb7/pt99+U05Ojm644QZJ0v79++Xu7q7GjRtb21ksFseMEgAA4Bqwddy0ItUPuKlo9QAcx64g1aVLFwUEBGjevHkKDi74PyIpKSnq16+fWrVqpWeeecahgwQAAACAksSue6SmTJmiCRMmWEOUJAUHB+v1119n1T4AAAAA1zy7glRqaqpOnjxZaP/Jkyd19uzZIg8KAAAAAEoyu6b2de3aVf369dOUKVN00003SZK2bNmiZ599Vt26dXPoAAEAAK4VHbq1ke/JE8qoGKIVX60zXf/y8jY6nXFCZX1D9HpH8/UAHMeuIDVr1iyNHDlSDz/8sHJycgo68vDQgAED9NZbbzl0gAAAANcK35Mn5JeUYHf96YwTSsmwvx6A49gVpPz8/PTee+/prbfe0qFDhyRJ1atXl7+/v0MHBwAAAAAlUZG+kPf48eM6fvy4atasKX9/fxmG4ahxAQAAAECJZVeQ+vvvv9WuXTvVqlVLnTp10vHjxyVJAwYMYOlzAAAAANc8u4LU8OHD5enpqfj4ePn5+Vn3P/jgg1q+fLnDBgcAAAAAJZFd90itXLlSK1asUOXKlW3216xZU3/++adDBgYAAAAAJZVdV6TOnTtncyXqvFOnTsnb27vIgwIAAACAksyuINWqVSvNnz/fum2xWJSfn69Jkybptttuc9jgAAAAAKAksmtq36RJk9SuXTv98ssvys7O1nPPPafff/9dp06d0saNGx09RgAAAAAoUewKUvXr19f+/fs1c+ZMBQQEKC0tTd26ddOQIUMUHh7u6DECAABcE357bow8MjKU6+trV/1DjcYoOzdDXh721QNwHNNBKicnRx07dtSsWbP00ksvOWNMAAAA16Q/u9xfpPoWVYtWD8BxTN8j5enpqZ07dzpjLAAAAABQKti12ESvXr300UcfOXosAAAAAFAq2HWPVG5urj7++GOtXr1aTZo0kb+/v83xqVOnOmRwAAAA15KAPw7ILS9X+e4eOlutpun6hNQDys/PlZubhyICzdcDcBxTQeqPP/5Q1apVtXv3bjVu3FiStH//fps2FovFcaMDAAC4hrTrc4/8khKUHhqhbzbsMV0/fs09SslIULBvhGZ2NV8PwHFMBamaNWvq+PHj+v777yVJDz74oKZPn67Q0FCnDA4AAAAASiJT90gZhmGzvWzZMp07d86hAwIAAACAks6uxSbO+3ewAgAAAIDrgakgZbFYCt0DxT1RAAAAAK43pu6RMgxDffv2lbe3tyQpMzNTTzzxRKFV+7766ivHjRAAAAAAShhTV6T69OmjkJAQBQUFKSgoSL169VJERIR1+/zDHm+++aYsFouGDRtm3ZeZmakhQ4aofPnyKlOmjLp3766kpCSbuvj4eHXu3Fl+fn4KCQnRs88+q9zcXLvGAAAAAABXw9QVqTlz5jhlEFu3btUHH3yghg0b2uwfPny4li5dqkWLFikoKEhDhw5Vt27dtHHjRklSXl6eOnfurLCwMG3atEnHjx9X79695enpqfHjxztlrAAAAABQpMUmHCEtLU09e/bUhx9+qODgYOv+M2fO6KOPPtLUqVPVtm1bNWnSRHPmzNGmTZv0008/SZJWrlypPXv26NNPP1VMTIzuvPNOjRs3Tu+++66ys7Nd9ZQAAAAAXONcHqSGDBmizp07q3379jb7t23bppycHJv9tWvXVmRkpDZv3ixJ2rx5sxo0aGDzPVYdOnRQamqqfv/990ueMysrS6mpqTYPAAAAALhapqb2OdrChQv166+/auvWrYWOJSYmysvLS2XLlrXZHxoaqsTERGubf38Z8Pnt820uZsKECRozZkwRRw8AAGDO8v+ulSU/T4abu1314zquVb6RJzeLffUAHMdlQero0aN6+umntWrVKvn4+BTruUeNGqURI0ZYt1NTU1WlSpViHQMAALj+ZIaEFak+2Ldo9QAcx2VT+7Zt26YTJ06ocePG8vDwkIeHh9avX6/p06fLw8NDoaGhys7O1unTp23qkpKSFBZW8CESFhZWaBW/89vn21yMt7e3AgMDbR4AAAAAcLVcFqTatWunXbt2afv27dZH06ZN1bNnT+vPnp6eWrNmjbVm3759io+PV2xsrCQpNjZWu3bt0okTJ6xtVq1apcDAQNWtW7fYnxMAAACA64PLpvYFBASofv36Nvv8/f1Vvnx56/4BAwZoxIgRKleunAIDA/Xkk08qNjZWN998syTpjjvuUN26dfXII49o0qRJSkxM1Msvv6whQ4ZYvzQYAACgpKi+cK4809OU41dGh3r0NV2/9uBcZeakycezjNrWMF8PwHFcutjElbz99ttyc3NT9+7dlZWVpQ4dOui9996zHnd3d9eSJUs0aNAgxcbGyt/fX3369NHYsWNdOGoAAICLa/DuJPklJSg9NMKuIPXVrklKyUhQsG/EdR+k4uLinNJvhQoVFBkZ6ZS+cW0pUUFq3bp1Nts+Pj5699139e67716yJioqSt99952TRwYAAICS4PTJJFksFvXq1csp/fv6+WlvXBxhCldUooIUAAAAcDnpqWdkGIb6jZuu6vUbOrTvhEP79d7IgUpOTiZI4YoIUgAAACh1wqNrKLpejKuHgeuYy1btAwAAAIDSiiAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAk1i1DwAAoJikRldXdkCgMitUtKs+PLC6/LwCFeRjXz0AxyFIAQAAFJO18/9XpPqX2hWtHoDjMLUPAAAAAEwiSAEAAACASQQpAAAAADCJe6QAAACKyS3PPCbvlL+VFVxem6Z8aLr+3Y2P6WzW3wrwLq8hLczXA3AcghQAAEAxCfl5o/ySEpQeGmFXfdyJjUrJSFCwr331AByHqX0AAAAAYBJBCgAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAk/hCXgAArgHx8fFKTk52St9xcXFO6fd6dPCB3vI6m6rsgEC76m+r0VsZ2any9bKvHoDjEKQAACjl4uPjVbtOHWWkpzv1PNlZ2U7t/3qw+8kXilTfvUHR6gE4DkEKAIBSLjk5WRnp6Ro8ebYiqtdyeP871q/SomlvKDc31+F9A0BpRZACAOAaEVG9lqLrxTi834RD+x3eJwCUdiw2AQAAAAAmcUUKAACgmNzbqq78khKUHhqhbzbsMV0/9Ou6SslIULBvhGZ2NV8PwHG4IgUAAAAAJhGkAAAAAMAkghQAAAAAmESQAgAAAACTCFIAAAAAYBJBCgAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACZ5uHoAAAAA14tNkz+Qe3a28ry87KoffMsHys3Lloe7ffUAHIcgBQAAUExONG9VpPq6oUWrB+A4TO0DAAAAAJMIUgAAAABgElP7AAAAiknIlg3We6Tsmea3J2mD9R4ppvkBrkWQAgAAKCa3jHxcfkkJSg+N0Dcb9piuf2/T40rJSFCwb4RmdjVfD8BxmNoHAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAkwhSAAAAAGASQQoAAAAATCJIAQAAAIBJBCkAAAAAMIkgBQAAAAAmebh6AAAAANeLbzbsKVL9zK5FqwfgOFyRAgAAAACTCFIAAAAAYBJBCgAAAABM4h4pAACAYlJ/xpvyOpuq7IBA7X7yBdP1/931pjKyU+XrFajuDczXA3AcghQAAEAxqfHFfPklJSg9NMKuIPX9wflKyUhQsG8EQQpwMab2AQAAAIBJBCkAAAAAMIkgBQAAAAAmEaQAAAAAwCSCFAAAAACYRJACAAAAAJMIUgAAAABgEkEKAAAAAEziC3kBAACKyYmbWsg75W9lBZe3q75OSAudzfpbAd721QNwHIIUAABAMdk05cMi1Q9pUbR6AI7D1D4AAAAAMIkgBQAAAAAmEaQAAAAAwCTukQIAACgmbXt3kU/ySWVWqKi18/9nuv6NNV10JvOkgnwq6qV25usBOA5BCgAAoJgEHj4kv6QEpZ9Ntav+eOohpWQkKD3bvnoAjsPUPgAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAkwhSAAAAAGASQQoAAAAATCJIAQAAAIBJfCEvAABAMdk15Dl5pqcpx6+MXfXdGjynzJw0+XjaVw/AcQhSAAAAxeRQj75Fqm9bo2j1AByHqX0AAAAAYBJBCgAAAABMcmmQmjBhgpo1a6aAgACFhITo3nvv1b59+2zaZGZmasiQISpfvrzKlCmj7t27KykpyaZNfHy8OnfuLD8/P4WEhOjZZ59Vbm5ucT4VAACAK/I5kSjfxGPyOZFoV31KRqL+Tj+mlAz76gE4jkvvkVq/fr2GDBmiZs2aKTc3Vy+++KLuuOMO7dmzR/7+/pKk4cOHa+nSpVq0aJGCgoI0dOhQdevWTRs3bpQk5eXlqXPnzgoLC9OmTZt0/Phx9e7dW56enho/frwrnx4AADbi4+OVnJzs8H7j4uIc3ieco2P3tvJLSlB6aIS+2bDHdP0ry9sqJSNBwb4RmtnVfD0Ax3FpkFq+fLnN9ty5cxUSEqJt27bp1ltv1ZkzZ/TRRx9pwYIFatu2rSRpzpw5qlOnjn766SfdfPPNWrlypfbs2aPVq1crNDRUMTExGjdunJ5//nmNHj1aXl5ehc6blZWlrKws63ZqaqpznygA4LoXHx+v2nXqKCM93WnnyM7KdlrfAABbJWrVvjNnzkiSypUrJ0natm2bcnJy1L59e2ub2rVrKzIyUps3b9bNN9+szZs3q0GDBgoNDbW26dChgwYNGqTff/9djRo1KnSeCRMmaMyYMU5+NgAAXJCcnKyM9HQNnjxbEdVrObTvHetXadG0N5jWDgDFqMQEqfz8fA0bNkwtWrRQ/fr1JUmJiYny8vJS2bJlbdqGhoYqMTHR2uafIer88fPHLmbUqFEaMWKEdTs1NVVVqlRx1FMBAOCSIqrXUnS9GIf2mXBov0P7AwBcWYkJUkOGDNHu3bv1448/Ov1c3t7e8vb2dvp5AAAAAFybSsTy50OHDtWSJUv0/fffq3Llytb9YWFhys7O1unTp23aJyUlKSwszNrm36v4nd8+3wYAAAAAHMmlQcowDA0dOlRff/211q5dq+joaJvjTZo0kaenp9asWWPdt2/fPsXHxys2NlaSFBsbq127dunEiRPWNqtWrVJgYKDq1q1bPE8EAAAAwHXFpVP7hgwZogULFujbb79VQECA9Z6moKAg+fr6KigoSAMGDNCIESNUrlw5BQYG6sknn1RsbKxuvvlmSdIdd9yhunXr6pFHHtGkSZOUmJiol19+WUOGDGH6HgAAAACncGmQev/99yVJbdq0sdk/Z84c9e3bV5L09ttvy83NTd27d1dWVpY6dOig9957z9rW3d1dS5Ys0aBBgxQbGyt/f3/16dNHY8eOLa6nAQAAAOA649IgZRjGFdv4+Pjo3Xff1bvvvnvJNlFRUfruu+8cOTQAAAAAuKQSs2ofAADAtW7NvG/llperfHf7fgV7sd23ys/PlZsbv8IBrsa/QgAAgGJytlrNItVHBBatHoDjlIjlzwEAAACgNCFIAQAAAIBJTO0DAAAoJlH/WySPjAzl+vrqzy73m67feGSRsnMz5OXhqxZVzdcDcByCFAAAQDFpNOk1+SUlKD00wq4g9Z/fXlNKRoKCfSMIUk4UFxfntL4rVKigyMhIp/WP4kOQAgAAACSdPpkki8WiXr16Oe0cvn5+2hsXR5i6BhCkAAAAAEnpqWdkGIb6jZuu6vUbOrz/hEP79d7IgUpOTiZIXQMIUgAAAMA/hEfXUHS9GFcPAyUcq/YBAAAAgEkEKQAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJjEqn0AAADFJKNiiM2fZpX1DbH5E4DrEKQAAACKyYqv1hWp/vWORasH4DhM7QMAAAAAkwhSAAAAAGASQQoAAAAATOIeKQAAgGLS7JVh8j6ToqygYG0dN810/Uc/D1NaVorKeAdrwE3m6wE4DkEKAACgmFRat1J+SQlKD43QVjvqfzu2UikZCQr2jXD42ACYw9Q+AAAAADCJIAUAAAAAJhGkAAAAAMAkghQAAAAAmESQAgAAAACTCFIAAAAAYBJBCgAAAABMIkgBAAAAgEl8IS8AAEAxOXJXd3mdOa3soLJ21d9StbvOZZ+Wv5d99QAchyAFAABQTLY/P65I9Q83Klo9AMdhah8AAAAAmESQAgAAAACTCFIAAAAAYBL3SAEAABSTzh2aye9EotJDwrR0xVbT9SOXNFNKeqKC/cI0+S7z9QAchyAFAMD/Fx8fr+TkZKf0HRcX55R+Ubp4pp+T57mz8kwPsKs+M+ecMnPPKjPHvnoAjkOQAgBABSGqdp06ykhPd+p5srOyndo/AKB4EKQAAJCUnJysjPR0DZ48WxHVazm8/x3rV2nRtDeUm5vr8L4BAMWPIAUAwD9EVK+l6HoxDu834dB+h/cJAHAdVu0DAAAAAJMIUgAAAABgEkEKAAAAAEwiSAEAAACASQQpAAAAADCJVfsAAACKyc9jp8o9M1N5Pj521fe/aapy8jLl6W5fPQDHIUgBAAAUk4TbOhapvnGlotUDcBym9gEAAACASQQpAAAAADCJqX0AAADFJHj3drnnZCvP00sp9WNM1x8+tV25ednycPdSdDnz9QAchyAFAABQTFoPelh+SQlKD43QNxv2mK6fsv5hpWQkKNg3QjO7mq8H4DhM7QMAAAAAk7giBQAAABSjuLg4p/RboUIFRUZGOqVvFEaQAgAAAIrB6ZNJslgs6tWrl1P69/Xz0964OMJUMSFIAQAAAMUgPfWMDMNQv3HTVb1+Q4f2nXBov94bOVDJyckEqWJCkAIAAACKUXh0DUXXi3H1MFBELDYBAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAk1hsAgAAoJgsWbZFkiHJYlf9W3dtkSFDFjvrATgOQQoAAKCY5JYJKFK9r2fR6gE4DkEKAFCqxMfHKzk52eH9xsXFObxPAMC1iyAFACg14uPjVbtOHWWkpzvtHNlZ2U7rGwBw7SBIAQBKjeTkZGWkp2vw5NmKqF7LoX3vWL9Ki6a9odzcXIf2C/xT7Y9nyjPtrHLKBGhv/6Gm67+Lm6mMnLPy9QxQpzrm6wE4DkEKAFDqRFSvpeh6MQ7tM+HQfof2B1xM7TnvyS8pQemhEfYFqb3vKSUjQcG+EQQpwMVY/hwAAAAATCJIAQAAAIBJBCkAAAAAMIkgBQAAAAAmEaQAAAAAwCSCFAAAAACYRJACAAAAAJMIUgAAAABgEl/ICwAAUExO1Wuo9PBKyixX3q766HINVT6zkgJ97KsH4DgEKQAAgGLyw6yFRap/pnXR6gE4DkEKAOBQ8fHxSk5OdkrfcXFxTukXAACzCFIAAIeJj49X7Tp1lJGe7tTzZGdlO7V/ACitnPk/nCpUqKDIyEin9V/aEKQAAA6TnJysjPR0DZ48WxHVazm8/x3rV2nRtDeUm5vr8L4BoDQ7fTJJFotFvXr1cto5fP38tDcujjD1/xGkAAAOF1G9lqLrxTi834RD+x3eJ1Ccbn2ih3xO/a3McuXtul9qyvoeSs38W4E+5blfCjbSU8/IMAz1Gzdd1es3dHj/CYf2672RA5WcnEyQ+v8IUgAAAMWk3O875ZeUoPTQCLvqD5/aqZSMBAX72lePa194dA2n/I8sFMb3SAEAAACASVyRAoDrkLNW1mNVPQDA9YIgBeCynLmUtVS6VwBy5mvjzNelOFbWY1U9AMC1jiAF4JKK4xfu0roCkLNfG2e+Ls5cWY9V9QAA14trJki9++67euutt5SYmKgbb7xRM2bM0E033eTqYQGlmrOXsi7NKwA587U5/7ps2LBBderUcWjf0oXpd85YWY9V9QAA14trIkh9/vnnGjFihGbNmqXmzZtr2rRp6tChg/bt26eQkBBXDw8o9Zy1lPV5zrqvpjimDTrjtSmO7wKRmH4HADCvNP8329GuiSA1depUPfbYY+rXr58kadasWVq6dKk+/vhjvfDCCy4enXnOvO8iKytL3t7eTulbcv59HaX1Xh1nj91Zf6/OXjjA2YGhtE4bdPZ3gTD9DgBgFv/NLqzUB6ns7Gxt27ZNo0aNsu5zc3NT+/bttXnz5ovWZGVlKSsry7p95swZSVJqaqpzB3sVjh49qqbNmikzI8NJZ7BIMpzUt+Tt46NP5s9XaGioQ/tNSkpS7969lZmZ6dB+/6k0j93Zf68Htv+izPRzDu/34G9bZRiGbn/kcYVHVXNo338n/qWl/zdDK1as0A033ODQviVp3759kqQjv+9w+GtzfnpcdmaGU173nP//+fdn3C65WRz7vjk/dmf07ez+Gbtr+k88fFCStG3bNqWlpTm0b6l4/q2aeV1uzc5UrqTU7EzFbd1ouv/ctEwpR8rNu3K9o8deUvovrX07u39nj704/pt95MgRlS1b1qF92+N8JjCMy7+OFuNKLUq4hIQEVapUSZs2bVJsbKx1/3PPPaf169dry5YthWpGjx6tMWPGFOcwAQAAAJQiR48eVeXKlS95vNRfkbLHqFGjNGLECOt2fn6+Tp06pfLly8tisbhwZNee1NRUValSRUePHlVgYKCrhwMX4r0AifcBCvA+wHm8FyCVvPeBYRg6e/asIiIiLtuu1AepChUqyN3dXUlJSTb7k5KSFBYWdtEab2/vQveTlITLiNeywMDAEvEPA67HewES7wMU4H2A83gvQCpZ74OgoKArtnErhnE4lZeXl5o0aaI1a9ZY9+Xn52vNmjU2U/0AAAAAwFFK/RUpSRoxYoT69Omjpk2b6qabbtK0adN07tw56yp+AAAAAOBI10SQevDBB3Xy5Em9+uqrSkxMVExMjJYvX+7w1ddgnre3t1577TWnLrmO0oH3AiTeByjA+wDn8V6AVHrfB6V+1T4AAAAAKG6l/h4pAAAAAChuBCkAAAAAMIkgBQAAAAAmEaQAAAAAwCSCFIrs/fffV8OGDa1fohYbG6tly5ZZj7dp00YWi8Xm8cQTT7hwxCgOb775piwWi4YNG2bdl5mZqSFDhqh8+fIqU6aMunfvXujLtHHtudh7gc+F68Po0aML/T3Xrl3bepzPhOvDld4HfB5cX44dO6ZevXqpfPny8vX1VYMGDfTLL79YjxuGoVdffVXh4eHy9fVV+/btdeDAAReO+NKuieXP4VqVK1fWm2++qZo1a8owDM2bN0/33HOPfvvtN9WrV0+S9Nhjj2ns2LHWGj8/P1cNF8Vg69at+uCDD9SwYUOb/cOHD9fSpUu1aNEiBQUFaejQoerWrZs2btzoopHC2S71XpD4XLhe1KtXT6tXr7Zue3hc+NWDz4Trx+XeBxKfB9eLlJQUtWjRQrfddpuWLVumihUr6sCBAwoODra2mTRpkqZPn6558+YpOjpar7zyijp06KA9e/bIx8fHhaMvjCCFIuvSpYvN9htvvKH3339fP/30kzVI+fn5KSwszBXDQzFLS0tTz5499eGHH+r111+37j9z5ow++ugjLViwQG3btpUkzZkzR3Xq1NFPP/2km2++2VVDhpNc6r1wHp8L1wcPD4+L/j3zmXB9udT74Dw+D64PEydOVJUqVTRnzhzrvujoaOvPhmFo2rRpevnll3XPPfdIkubPn6/Q0FB988036tGjR7GP+XKY2geHysvL08KFC3Xu3DnFxsZa93/22WeqUKGC6tevr1GjRik9Pd2Fo4QzDRkyRJ07d1b79u1t9m/btk05OTk2+2vXrq3IyEht3ry5uIeJYnCp98J5fC5cHw4cOKCIiAhVq1ZNPXv2VHx8vCQ+E643l3ofnMfnwfVh8eLFatq0qe6//36FhISoUaNG+vDDD63HDx8+rMTERJvPhaCgIDVv3rxEfi5wRQoOsWvXLsXGxiozM1NlypTR119/rbp160qSHn74YUVFRSkiIkI7d+7U888/r3379umrr75y8ajhaAsXLtSvv/6qrVu3FjqWmJgoLy8vlS1b1mZ/aGioEhMTi2mEKC6Xey9IfC5cL5o3b665c+fqhhtu0PHjxzVmzBi1atVKu3fv5jPhOnK590FAQACfB9eRP/74Q++//75GjBihF198UVu3btVTTz0lLy8v9enTx/pvPzQ01KaupH4uEKTgEDfccIO2b9+uM2fO6Msvv1SfPn20fv161a1bVwMHDrS2a9CggcLDw9WuXTsdOnRI1atXd+Go4UhHjx7V008/rVWrVpW4OcwoXlfzXuBz4fpw5513Wn9u2LChmjdvrqioKH3xxRfy9fV14chQnC73PhgwYACfB9eR/Px8NW3aVOPHj5ckNWrUSLt379asWbPUp08fF4/OPKb2wSG8vLxUo0YNNWnSRBMmTNCNN96od95556JtmzdvLkk6ePBgcQ4RTrZt2zadOHFCjRs3loeHhzw8PLR+/XpNnz5dHh4eCg0NVXZ2tk6fPm1Tl5SUxLz4a8yV3gt5eXmFavhcuD6ULVtWtWrV0sGDBxUWFsZnwnXqn++Di+Hz4NoVHh5unbF0Xp06daxTPc//2//36p0l9XOBIAWnyM/PV1ZW1kWPbd++XVLBPyZcO9q1a6ddu3Zp+/bt1kfTpk3Vs2dP68+enp5as2aNtWbfvn2Kj4+3uZ8Opd+V3gvu7u6FavhcuD6kpaXp0KFDCg8PV5MmTfhMuE79831wMXweXLtatGihffv22ezbv3+/oqKiJBUsPBEWFmbzuZCamqotW7aUyM8FpvahyEaNGqU777xTkZGROnv2rBYsWKB169ZpxYoVOnTokBYsWKBOnTqpfPny2rlzp4YPH65bb731ossho/QKCAhQ/fr1bfb5+/urfPny1v0DBgzQiBEjVK5cOQUGBurJJ59UbGwsq3NdY670XuBz4foxcuRIdenSRVFRUUpISNBrr70md3d3PfTQQwoKCuIz4TpxufcBnwfXl+HDh+uWW27R+PHj9cADD+jnn3/W7NmzNXv2bEmyfufg66+/rpo1a1qXP4+IiNC9997r2sFfBEEKRXbixAn17t1bx48fV1BQkBo2bKgVK1bo9ttv19GjR7V69WpNmzZN586dU5UqVdS9e3e9/PLLrh42XODtt9+Wm5ubunfvrqysLHXo0EHvvfeeq4eFYubl5cXnwnXir7/+0kMPPaS///5bFStWVMuWLfXTTz+pYsWKkvhMuF5c7n2QmZnJ58F1pFmzZvr66681atQojR07VtHR0Zo2bZp69uxpbfPcc8/p3LlzGjhwoE6fPq2WLVtq+fLlJfL+a4thGIarBwEAAAAApQn3SAEAAACASQQpAAAAADCJIAUAAAAAJhGkAAAAAMAkghQAAAAAmESQAgAAAACTCFIAAAAAYBJBCgAAAABMIkgBAEqUI0eOyGKxaPv27a4eSonRpk0bDRs2zNXDAAD8A0EKAOBwFovlso/Ro0e7eoiFlISwsm7dOlksFp0+fdql4wAAXJmHqwcAALj2HD9+3Prz559/rldffVX79u2z7itTpowrhgUAgMNwRQoA4HBhYWHWR1BQkCwWi3U7JCREU6dOVeXKleXt7a2YmBgtX778kn3l5eWpf//+ql27tuLj4yVJ3377rRo3biwfHx9Vq1ZNY8aMUW5urrXGYrHo//7v/9S1a1f5+fmpZs2aWrx4cZGe048//qhWrVrJ19dXVapU0VNPPaVz585Zj1etWlXjx49X//79FRAQoMjISM2ePdumj02bNikmJkY+Pj5q2rSpvvnmG+s0xiNHjui2226TJAUHB8tisahv377W2vz8fD333HMqV66cwsLCSuRVPQC4nhCkAADF6p133tGUKVM0efJk7dy5Ux06dNDdd9+tAwcOFGqblZWl+++/X9u3b9eGDRsUGRmpDRs2qHfv3nr66ae1Z88effDBB5o7d67eeOMNm9oxY8bogQce0M6dO9WpUyf17NlTp06dsmvMhw4dUseOHdW9e3ft3LlTn3/+uX788UcNHTrUpt2UKVPUtGlT/fbbbxo8eLAGDRpkvRKXmpqqLl26qEGDBvr11181btw4Pf/889baKlWq6L///a8kad++fTp+/Ljeeecd6/F58+bJ399fW7Zs0aRJkzR27FitWrXKrucDAHAAAwAAJ5ozZ44RFBRk3Y6IiDDeeOMNmzbNmjUzBg8ebBiGYRw+fNiQZGzYsMFo166d0bJlS+P06dPWtu3atTPGjx9vU//JJ58Y4eHh1m1Jxssvv2zdTktLMyQZy5Ytu+Q4W7dubTz99NMXPTZgwABj4MCBNvs2bNhguLm5GRkZGYZhGEZUVJTRq1cv6/H8/HwjJCTEeP/99w3DMIz333/fKF++vLW9YRjGhx9+aEgyfvvtN8MwDOP77783JBkpKSmFxtayZUubfc2aNTOef/75Sz4fAIBzcY8UAKDYpKamKiEhQS1atLDZ36JFC+3YscNm30MPPaTKlStr7dq18vX1te7fsWOHNm7caHMFKi8vT5mZmUpPT5efn58kqWHDhtbj/v7+CgwM1IkTJ+wa944dO7Rz50599tln1n2GYSg/P1+HDx9WnTp1Cp3z/HTG8+fct2+fGjZsKB8fH2ubm2666arH8M++JSk8PNzu5wMAKDqCFACgROrUqZM+/fRTbd68WW3btrXuT0tL05gxY9StW7dCNf8MKZ6enjbHLBaL8vPz7RpLWlqaHn/8cT311FOFjkVGRjrlnP/mzL4BAOYRpAAAxSYwMFARERHauHGjWrdubd2/cePGQldnBg0apPr16+vuu+/W0qVLre0bN26sffv2qUaNGsU27saNG2vPnj1FOucNN9ygTz/9VFlZWfL29pYkbd261aaNl5eXpIIrbACAko0gBQAoVs8++6xee+01Va9eXTExMZozZ462b99uM23uvCeffFJ5eXm66667tGzZMrVs2VKvvvqq7rrrLkVGRuq+++6Tm5ubduzYod27d+v1118v0thOnjxZ6IuAw8PD9fzzz+vmm2/W0KFD9eijj8rf31979uzRqlWrNHPmzKvq++GHH9ZLL72kgQMH6oUXXlB8fLwmT54sqeDqkiRFRUXJYrFoyZIl6tSpk3x9fVkqHgBKKFbtAwAUq6eeekojRozQM888owYNGmj58uVavHixatasedH2w4YN05gxY9SpUydt2rRJHTp00JIlS7Ry5Uo1a9ZMN998s95++21FRUUVeWwLFixQo0aNbB4ffvihGjZsqPXr12v//v1q1aqVGjVqpFdffVURERFX3XdgYKD+97//afv27YqJidFLL72kV199VdKFKYmVKlXSmDFj9MILLyg0NLTQqoAAgJLDYhiG4epBAABwPfrss8/Ur18/nTlzxmZBDQBAycfUPgAAisn8+fNVrVo1VapUSTt27NDzzz+vBx54gBAFAKUQQQoAgGKSmJioV199VYmJiQoPD9f9999f6IuEAQClA1P7AAAAAMAkFpsAAAAAAJMIUgAAAABgEkEKAAAAAEwiSAEAAACASQQpAAAAADCJIAUAAAAAJhGkAAAAAMAkghQAAAAAmPT/ALFDWFiHUDxIAAAAAElFTkSuQmCC",
|
||
"text/plain": [
|
||
"<Figure size 1000x600 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"#统计图\n",
|
||
"import matplotlib.pyplot as plt\n",
|
||
"import seaborn as sns\n",
|
||
"import numpy as np\n",
|
||
"\n",
|
||
"# 假设这是您的 token_len_list\n",
|
||
"\n",
|
||
"# 设置画布大小\n",
|
||
"plt.figure(figsize=(10, 6))\n",
|
||
"\n",
|
||
"# 使用 seaborn 生成直方图\n",
|
||
"sns.histplot(token_len_list, bins=30, kde=False, color=\"skyblue\", edgecolor=\"black\")\n",
|
||
"\n",
|
||
"# 添加标题和标签\n",
|
||
"plt.title(\"Distribution of Token Lengths\")\n",
|
||
"plt.xlabel(\"Token Length\")\n",
|
||
"plt.ylabel(\"Frequency\")\n",
|
||
"\n",
|
||
"# 显示平均值线\n",
|
||
"mean_value = np.mean(token_len_list)\n",
|
||
"plt.axvline(mean_value, color='red', linestyle='dashed', linewidth=2)\n",
|
||
"plt.text(mean_value + 2, plt.ylim()[1]*0.9, f'Mean: {mean_value:.2f}', color='red')\n",
|
||
"\n",
|
||
"# 显示中位数线\n",
|
||
"median_value = np.median(token_len_list)\n",
|
||
"plt.axvline(median_value, color='green', linestyle='dashed', linewidth=2)\n",
|
||
"plt.text(median_value - 10, plt.ylim()[1]*0.8, f'Median: {median_value:.2f}', color='green')\n",
|
||
"\n",
|
||
"# 显示图形\n",
|
||
"plt.show()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 15,
|
||
"id": "9a65c8bc-6bf0-4605-8c38-409bbb14f2c7",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"application/vnd.jupyter.widget-view+json": {
|
||
"model_id": "a4e97d92506f419581c3711f26d7f683",
|
||
"version_major": 2,
|
||
"version_minor": 0
|
||
},
|
||
"text/plain": [
|
||
"Map: 0%| | 0/53275 [00:00<?, ? examples/s]"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"data": {
|
||
"application/vnd.jupyter.widget-view+json": {
|
||
"model_id": "9004c03cb9b24411b6bd9d33662402fb",
|
||
"version_major": 2,
|
||
"version_minor": 0
|
||
},
|
||
"text/plain": [
|
||
"Map: 0%| | 0/5920 [00:00<?, ? examples/s]"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"# 2. tokenize\n",
|
||
"def tokenize_function(examples):\n",
|
||
" examples['label'] = [int(item) for item in examples['label']]\n",
|
||
" return tokenizer(examples['sequence'], truncation=True, padding='max_length', max_length=128)\n",
|
||
"\n",
|
||
"# 3. 对数据集应用分词函数\n",
|
||
"tokenized_datasets = dataset.map(tokenize_function, batched=True)\n",
|
||
"\n",
|
||
"# 4. 创建一个数据收集器，用于动态填充和遮蔽\n",
|
||
"data_collator = DataCollatorWithPadding(tokenizer=tokenizer)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 22,
|
||
"id": "4b0faa94-d0c4-4ce8-9976-dcefcb766f0b",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"/root/miniconda3/lib/python3.12/site-packages/transformers/training_args.py:1575: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead\n",
|
||
" warnings.warn(\n",
|
||
"/tmp/ipykernel_2549/341301010.py:29: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `Trainer.__init__`. Use `processing_class` instead.\n",
|
||
" trainer = Trainer(\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"from transformers import TrainingArguments, Trainer\n",
|
||
"import numpy as np\n",
|
||
"import torch.nn as nn\n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"def compute_metrics(eval_pred):\n",
|
||
" predictions, labels = eval_pred\n",
|
||
" predictions = np.argmax(predictions, axis=1)\n",
|
||
" return {'accuracy': (predictions==labels).sum() / len(labels)}\n",
|
||
"\n",
|
||
"# change training hyperparameters to archive better quality\n",
|
||
"training_args = TrainingArguments(\n",
|
||
" output_dir=\"ds_job_category_v0\",\n",
|
||
" learning_rate=1e-5,\n",
|
||
" lr_scheduler_type=\"constant_with_warmup\",\n",
|
||
" warmup_ratio=0.1,\n",
|
||
" optim='adamw_torch',\n",
|
||
" weight_decay=0.0,\n",
|
||
" per_device_train_batch_size=20,\n",
|
||
" per_device_eval_batch_size=20,\n",
|
||
" num_train_epochs=10,\n",
|
||
" evaluation_strategy=\"epoch\",\n",
|
||
" save_strategy=\"epoch\",\n",
|
||
" logging_strategy=\"epoch\",\n",
|
||
" load_best_model_at_end=True\n",
|
||
")\n",
|
||
"\n",
|
||
"trainer = Trainer(\n",
|
||
" model=model,\n",
|
||
" args=training_args,\n",
|
||
" train_dataset=tokenized_datasets[\"train\"],\n",
|
||
" eval_dataset=tokenized_datasets[\"test\"],\n",
|
||
" tokenizer=tokenizer,\n",
|
||
" data_collator=data_collator,\n",
|
||
" compute_metrics=compute_metrics,\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 17,
|
||
"id": "9b067740-9c0f-4df8-a5af-b68ec9d1f3e0",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"\n",
|
||
" <div>\n",
|
||
" \n",
|
||
" <progress value='26640' max='26640' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
|
||
" [26640/26640 1:00:13, Epoch 10/10]\n",
|
||
" </div>\n",
|
||
" <table border=\"1\" class=\"dataframe\">\n",
|
||
" <thead>\n",
|
||
" <tr style=\"text-align: left;\">\n",
|
||
" <th>Epoch</th>\n",
|
||
" <th>Training Loss</th>\n",
|
||
" <th>Validation Loss</th>\n",
|
||
" <th>Accuracy</th>\n",
|
||
" </tr>\n",
|
||
" </thead>\n",
|
||
" <tbody>\n",
|
||
" <tr>\n",
|
||
" <td>1</td>\n",
|
||
" <td>0.324900</td>\n",
|
||
" <td>0.237557</td>\n",
|
||
" <td>0.916216</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <td>2</td>\n",
|
||
" <td>0.193100</td>\n",
|
||
" <td>0.212998</td>\n",
|
||
" <td>0.925338</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <td>3</td>\n",
|
||
" <td>0.126900</td>\n",
|
||
" <td>0.278650</td>\n",
|
||
" <td>0.923480</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <td>4</td>\n",
|
||
" <td>0.076900</td>\n",
|
||
" <td>0.362979</td>\n",
|
||
" <td>0.922804</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <td>5</td>\n",
|
||
" <td>0.047400</td>\n",
|
||
" <td>0.518552</td>\n",
|
||
" <td>0.915372</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <td>6</td>\n",
|
||
" <td>0.032000</td>\n",
|
||
" <td>0.698843</td>\n",
|
||
" <td>0.918412</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <td>7</td>\n",
|
||
" <td>0.029000</td>\n",
|
||
" <td>0.760331</td>\n",
|
||
" <td>0.915709</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <td>8</td>\n",
|
||
" <td>0.025900</td>\n",
|
||
" <td>0.769762</td>\n",
|
||
" <td>0.921959</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <td>9</td>\n",
|
||
" <td>0.021800</td>\n",
|
||
" <td>0.740165</td>\n",
|
||
" <td>0.923142</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <td>10</td>\n",
|
||
" <td>0.021300</td>\n",
|
||
" <td>0.738664</td>\n",
|
||
" <td>0.922973</td>\n",
|
||
" </tr>\n",
|
||
" </tbody>\n",
|
||
"</table><p>"
|
||
],
|
||
"text/plain": [
|
||
"<IPython.core.display.HTML object>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"TrainOutput(global_step=26640, training_loss=0.08990609108864724, metrics={'train_runtime': 3619.5996, 'train_samples_per_second': 147.185, 'train_steps_per_second': 7.36, 'total_flos': 3.4801460969472e+16, 'train_loss': 0.08990609108864724, 'epoch': 10.0})"
|
||
]
|
||
},
|
||
"execution_count": 17,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"trainer.train()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 20,
|
||
"id": "aa26e020-2dfd-4e0e-b330-250ee3e44a44",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [],
|
||
"text/plain": [
|
||
"<IPython.core.display.HTML object>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"{'accuracy': 0.9253378378378379, 'f1': 0.927062706270627}"
|
||
]
|
||
},
|
||
"execution_count": 20,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"#模型测试\n",
|
||
"import evaluate\n",
|
||
"predictions = trainer.predict(tokenized_datasets[\"test\"])\n",
|
||
"preds = np.argmax(predictions.predictions, axis=-1)\n",
|
||
"metric = evaluate.load(\"glue\", \"mrpc\")\n",
|
||
"ret = metric.compute(predictions=preds, references=predictions.label_ids)\n",
|
||
"ret"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 21,
|
||
"id": "5e6d99ad-66a0-4b85-9380-ae2b7ee88056",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAHHCAYAAACcHAM1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy80BEi2AAAACXBIWXMAAA9hAAAPYQGoP6dpAABRWklEQVR4nO3deVgV1f8H8PeAclkv4AKIEoKCguJuRrgmgYpbWGaSool9LdDc0MwNtyzcUnMpS1HT1Eot0VQUd3FNlFxIEEUTsERAUPb5/eGPySt45crFO17er555Hu7MmTOfuQ/px885Z0YQRVEEERERkYwZ6DoAIiIiomdhwkJERESyx4SFiIiIZI8JCxEREckeExYiIiKSPSYsREREJHtMWIiIiEj2mLAQERGR7DFhISIiItljwkL0Erp69Sp8fHxgaWkJQRCwfft2rfZ//fp1CIKAiIgIrfb7MuvcuTM6d+6s6zCIqiwmLETPKTExEf/73//g7OwMY2NjKJVKeHl5YfHixXj48GGlXjswMBBxcXGYM2cO1q9fjzZt2lTq9V6kIUOGQBAEKJXKMr/Hq1evQhAECIKA+fPna9z/7du3ERYWhtjYWC1ES0QvSjVdB0D0Mtq5cyfeeecdKBQKDB48GE2bNkV+fj6OHj2K0NBQXLx4Ed9++22lXPvhw4eIiYnB5MmTERISUinXcHR0xMOHD1G9evVK6f9ZqlWrhgcPHmDHjh3o37+/yrENGzbA2NgYubm5z9X37du3MWPGDNSvXx8tWrQo93l79+59rusRkXYwYSHSUFJSEgYMGABHR0dER0ejTp060rHg4GAkJCRg586dlXb9f/75BwBgZWVVadcQBAHGxsaV1v+zKBQKeHl54ccffyyVsGzcuBF+fn745ZdfXkgsDx48gKmpKYyMjF7I9YiobBwSItJQeHg4srOz8f3336skKyUaNmyITz75RPpcWFiIWbNmoUGDBlAoFKhfvz4+++wz5OXlqZxXv3599OzZE0ePHsWrr74KY2NjODs7Y926dVKbsLAwODo6AgBCQ0MhCALq168P4NFQSsnPjwsLC4MgCCr7oqKi0L59e1hZWcHc3ByNGjXCZ599Jh1/2hyW6OhodOjQAWZmZrCyskKfPn1w+fLlMq+XkJCAIUOGwMrKCpaWlhg6dCgePHjw9C/2CQMHDsTvv/+OjIwMad/p06dx9epVDBw4sFT79PR0jB8/Hh4eHjA3N4dSqUT37t1x/vx5qc3BgwfRtm1bAMDQoUOloaWS++zcuTOaNm2Ks2fPomPHjjA1NZW+lyfnsAQGBsLY2LjU/fv6+sLa2hq3b98u970S0bMxYSHS0I4dO+Ds7IzXX3+9XO2DgoIwbdo0tGrVCosWLUKnTp0wd+5cDBgwoFTbhIQEvP3223jzzTexYMECWFtbY8iQIbh48SIAwN/fH4sWLQIAvPfee1i/fj2++uorjeK/ePEievbsiby8PMycORMLFixA7969cezYMbXn7du3D76+vrhz5w7CwsIwduxYHD9+HF5eXrh+/Xqp9v3798f9+/cxd+5c9O/fHxEREZgxY0a54/T394cgCNi6dau0b+PGjWjcuDFatWpVqv21a9ewfft29OzZEwsXLkRoaCji4uLQqVMnKXlwc3PDzJkzAQAffvgh1q9fj/Xr16Njx45SP3fv3kX37t3RokULfPXVV+jSpUuZ8S1evBi1a9dGYGAgioqKAADffPMN9u7di6VLl8Le3r7c90pE5SASUbllZmaKAMQ+ffqUq31sbKwIQAwKClLZP378eBGAGB0dLe1zdHQUAYiHDx+W9t25c0dUKBTiuHHjpH1JSUkiAHHevHkqfQYGBoqOjo6lYpg+fbr4+P/qixYtEgGI//zzz1PjLrnGmjVrpH0tWrQQbWxsxLt370r7zp8/LxoYGIiDBw8udb0PPvhApc+33npLrFmz5lOv+fh9mJmZiaIoim+//bbYtWtXURRFsaioSLSzsxNnzJhR5neQm5srFhUVlboPhUIhzpw5U9p3+vTpUvdWolOnTiIAceXKlWUe69Spk8q+PXv2iADE2bNni9euXRPNzc3Fvn37PvMeiUhzrLAQaSArKwsAYGFhUa72u3btAgCMHTtWZf+4ceMAoNRcF3d3d3To0EH6XLt2bTRq1AjXrl177pifVDL35ddff0VxcXG5zklJSUFsbCyGDBmCGjVqSPubNWuGN998U7rPx40YMULlc4cOHXD37l3pOyyPgQMH4uDBg0hNTUV0dDRSU1PLHA4CHs17MTB49EdaUVER7t69Kw13/fHHH+W+pkKhwNChQ8vV1sfHB//73/8wc+ZM+Pv7w9jYGN988025r0VE5ceEhUgDSqUSAHD//v1ytb9x4wYMDAzQsGFDlf12dnawsrLCjRs3VPa/8sorpfqwtrbGvXv3njPi0t599114eXkhKCgItra2GDBgALZs2aI2eSmJs1GjRqWOubm54d9//0VOTo7K/ifvxdraGgA0upcePXrAwsICmzdvxoYNG9C2bdtS32WJ4uJiLFq0CC4uLlAoFKhVqxZq166NCxcuIDMzs9zXrFu3rkYTbOfPn48aNWogNjYWS5YsgY2NTbnPJaLyY8JCpAGlUgl7e3v8+eefGp335KTXpzE0NCxzvyiKz32NkvkVJUxMTHD48GHs27cPgwYNwoULF/Duu+/izTffLNW2IipyLyUUCgX8/f2xdu1abNu27anVFQD4/PPPMXbsWHTs2BE//PAD9uzZg6ioKDRp0qTclSTg0fejiXPnzuHOnTsAgLi4OI3OJaLyY8JCpKGePXsiMTERMTExz2zr6OiI4uJiXL16VWV/WloaMjIypBU/2mBtba2yoqbEk1UcADAwMEDXrl2xcOFCXLp0CXPmzEF0dDQOHDhQZt8lccbHx5c6duXKFdSqVQtmZmYVu4GnGDhwIM6dO4f79++XOVG5xM8//4wuXbrg+++/x4ABA+Dj4wNvb+9S30l5k8fyyMnJwdChQ+Hu7o4PP/wQ4eHhOH36tNb6J6L/MGEh0tCECRNgZmaGoKAgpKWllTqemJiIxYsXA3g0pAGg1EqehQsXAgD8/Py0FleDBg2QmZmJCxcuSPtSUlKwbds2lXbp6emlzi15gNqTS61L1KlTBy1atMDatWtVEoA///wTe/fule6zMnTp0gWzZs3C119/DTs7u6e2MzQ0LFW9+emnn/D333+r7CtJrMpK7jQ1ceJEJCcnY+3atVi4cCHq16+PwMDAp36PRPT8+OA4Ig01aNAAGzduxLvvvgs3NzeVJ90eP34cP/30E4YMGQIAaN68OQIDA/Htt98iIyMDnTp1wqlTp7B27Vr07dv3qUtmn8eAAQMwceJEvPXWWxg1ahQePHiAFStWwNXVVWXS6cyZM3H48GH4+fnB0dERd+7cwfLly1GvXj20b9/+qf3PmzcP3bt3h6enJ4YNG4aHDx9i6dKlsLS0RFhYmNbu40kGBgaYMmXKM9v17NkTM2fOxNChQ/H6668jLi4OGzZsgLOzs0q7Bg0awMrKCitXroSFhQXMzMzQrl07ODk5aRRXdHQ0li9fjunTp0vLrNesWYPOnTtj6tSpCA8P16g/InoGHa9SInpp/fXXX+Lw4cPF+vXri0ZGRqKFhYXo5eUlLl26VMzNzZXaFRQUiDNmzBCdnJzE6tWriw4ODuKkSZNU2ojio2XNfn5+pa7z5HLapy1rFkVR3Lt3r9i0aVPRyMhIbNSokfjDDz+UWta8f/9+sU+fPqK9vb1oZGQk2tvbi++99574119/lbrGk0t/9+3bJ3p5eYkmJiaiUqkUe/XqJV66dEmlTcn1nlw2vWbNGhGAmJSU9NTvVBRVlzU/zdOWNY8bN06sU6eOaGJiInp5eYkxMTFlLkf+9ddfRXd3d7FatWoq99mpUyexSZMmZV7z8X6ysrJER0dHsVWrVmJBQYFKuzFjxogGBgZiTEyM2nsgIs0IoqjBDDgiIiIiHeAcFiIiIpI9JixEREQke0xYiIiISPaYsBAREZHsMWEhIiIi2WPCQkRERLLHB8dVsuLiYty+fRsWFhZafSQ4ERG9GKIo4v79+7C3t5feCK5tubm5yM/P10pfRkZGMDY21kpfcsKEpZLdvn0bDg4Oug6DiIgq6ObNm6hXr57W+83NzYWJRU2g8IFW+rOzs0NSUpLeJS1MWCqZhYUFAMCo7WgI1RQ6joaoctzYMUnXIRBVmvv3s+Di9Ir057m25efnA4UPoHAPBAyNKtZZUT5SL61Ffn4+ExbSTMkwkFBNwYSF9JZSqdR1CESVrtKH9asZQ6hgwiIK+js1lQkLERGRHAgAKpoU6fFUSSYsREREciAYPNoq2oee0t87IyIiIr3BCgsREZEcCIIWhoT0d0yICQsREZEccEhILf29MyIiItIbrLAQERHJAYeE1GLCQkREJAtaGBLS44ET/b0zIiIi0hussBAREckBh4TUYsJCREQkB1wlpJb+3hkRERHpDVZYiIiI5IBDQmoxYSEiIpIDDgmpxYSFiIhIDlhhUUt/UzEiIiLSG6ywEBERyQGHhNRiwkJERCQHgqCFhIVDQkREREQ6wwoLERGRHBgIj7aK9qGnmLAQERHJAeewqKW/d0ZERER6gxUWIiIiOeBzWNRiwkJERCQHHBJSS3/vjIiIiPQGKyxERERywCEhtZiwEBERyQGHhNRiwkJERCQHrLCopb+pGBEREekNVliIiIjkgENCajFhISIikgMOCamlv6kYERER6Q1WWIiIiGRBC0NCelyHYMJCREQkBxwSUkt/UzEiIiLSG6ywEBERyYEgaGGVkP5WWJiwEBERyQGXNaulv3dGREREas2dOxdt27aFhYUFbGxs0LdvX8THx6u06dy5MwRBUNlGjBih0iY5ORl+fn4wNTWFjY0NQkNDUVhYqNLm4MGDaNWqFRQKBRo2bIiIiAiNYmXCQkREJAclk24rumng0KFDCA4OxokTJxAVFYWCggL4+PggJydHpd3w4cORkpIibeHh4dKxoqIi+Pn5IT8/H8ePH8fatWsRERGBadOmSW2SkpLg5+eHLl26IDY2FqNHj0ZQUBD27NlT7lg5JERERCQHOhgS2r17t8rniIgI2NjY4OzZs+jYsaO039TUFHZ2dmX2sXfvXly6dAn79u2Dra0tWrRogVmzZmHixIkICwuDkZERVq5cCScnJyxYsAAA4ObmhqNHj2LRokXw9fUtV6yssBAREcmBFissWVlZKlteXl65QsjMzAQA1KhRQ2X/hg0bUKtWLTRt2hSTJk3CgwcPpGMxMTHw8PCAra2ttM/X1xdZWVm4ePGi1Mbb21ulT19fX8TExJT762GFhYiISM84ODiofJ4+fTrCwsLUnlNcXIzRo0fDy8sLTZs2lfYPHDgQjo6OsLe3x4ULFzBx4kTEx8dj69atAIDU1FSVZAWA9Dk1NVVtm6ysLDx8+BAmJibPvCcmLERERHKgxSGhmzdvQqlUSrsVCsUzTw0ODsaff/6Jo0ePquz/8MMPpZ89PDxQp04ddO3aFYmJiWjQoEHF4tUAh4SIiIjkQItDQkqlUmV7VsISEhKCyMhIHDhwAPXq1VPbtl27dgCAhIQEAICdnR3S0tJU2pR8Lpn38rQ2SqWyXNUVgAkLERFRlSWKIkJCQrBt2zZER0fDycnpmefExsYCAOrUqQMA8PT0RFxcHO7cuSO1iYqKglKphLu7u9Rm//79Kv1ERUXB09Oz3LEyYSEiIpKBJ5918rybJoKDg/HDDz9g48aNsLCwQGpqKlJTU/Hw4UMAQGJiImbNmoWzZ8/i+vXr+O233zB48GB07NgRzZo1AwD4+PjA3d0dgwYNwvnz57Fnzx5MmTIFwcHBUmVnxIgRuHbtGiZMmIArV65g+fLl2LJlC8aMGVPuWJmwEBERyYAuEpYVK1YgMzMTnTt3Rp06daRt8+bNAAAjIyPs27cPPj4+aNy4McaNG4d+/fphx44dUh+GhoaIjIyEoaEhPD098f7772Pw4MGYOXOm1MbJyQk7d+5EVFQUmjdvjgULFuC7774r95JmgJNuiYiIqixRFNUed3BwwKFDh57Zj6OjI3bt2qW2TefOnXHu3DmN4nscExYiIiI5EP5/q2gfeooJCxERkQw8z5BOGZ1oJxgZ4hwWIiIikj1WWIiIiGSAFRb1mLAQERHJABMW9ZiwEBERyQATFvU4h4WIiIhkjxUWIiIiOeCyZrWYsBAREckAh4TU45AQERERyR4rLERERDIgCNBChUU7scgRExYiIiIZEKCFISE9zlg4JERERESyxwoLERGRDHDSrXpMWIiIiOSAy5rV4pAQERERyR4rLERERHKghSEhkUNCREREVJm0MYel4quM5IsJCxERkQwwYVGPc1iIiIhI9lhhISIikgOuElKLCQsREZEMcEhIPQ4JERERkeyxwkJERCQDrLCox4SFiIhIBpiwqMchISIiIpI9VliIiIhkgBUW9ZiwEBERyQGXNavFISEiIiKSPVZYiIiIZIBDQuoxYSEiIpIBJizqMWEhIiKSASYs6nEOCxEREckeKyxERERywFVCajFhISIikgEOCanHISEiIiKSvZeiwiIIArZt24a+ffvqOhR6Aca81wE9O7jB5ZVayM0rwKmLNxG2KgoJN++qtGvrXg9ThnVF68b1UFRcjD8TU9Fvwnrk5hdKbXzauSB0cGc0cbZFXn4hjp2/jvenbZKOd2zphMkfvAE3J1s8yM3Hpj3nMev7/SgqLn5h90u0KGIvIg+cx9UbaTBWVMerHk6YPrIPXBxtAQD3MnPwxbe7cODkFdxKu4eaVubw69QMn43wg9LcBACwMfIEQmZuKLP/+N2fo3YNixd2P/R8WGFRT+cJS2pqKubMmYOdO3fi77//ho2NDVq0aIHRo0eja9euug4Poihi+vTpWLVqFTIyMuDl5YUVK1bAxcVF16HprdebO+K7X0/hXPzfqGZggKlB3tgaPhivDf0aD3ILADxKVn7+YhAW/XgEE5fuQmFRMZo626FYFKV+enVww+JxvTHr+/04fC4J1QwN4FbfRjre1NkWW+a+jwUbDmPE3G2oU8sCC8f0goGhgGkr977w+6aq69gfCRj2Tge0dHNEUVERZq3YgX4jlyFm82SYmSiQ8m8mUv7NxMxP+qKRkx1upqRj3BebkfJvJtZ+MQwA8JZ3K3R9zV2l3+CZPyAvv4DJyktCgBYSFj2exKLThOX69evw8vKClZUV5s2bBw8PDxQUFGDPnj0IDg7GlStXdBkeACA8PBxLlizB2rVr4eTkhKlTp8LX1xeXLl2CsbGxrsPTS+98+oPK54+/3IaEbRPRwtUexy/cAADM+bgbvtl2El/9eFRq93gFxtDAAHNDumPaN1H44fc/pP3xN/6Rfn6rS1NcvJaGeesPAQCSbqcj7Nu9WD2tP8LXHkT2w/xKuT+iJ/285GOVz8umvQ9X389w/vJNvN6qIdwb2GPdl0HScad6tTH5o14YMX0dCguLUK2aIUyMjWBibCS1+ffefRw58xeWTBn4wu6DqDLpdA7Lxx9/DEEQcOrUKfTr1w+urq5o0qQJxo4dixMnTjz1vIkTJ8LV1RWmpqZwdnbG1KlTUVBQIB0/f/48unTpAgsLCyiVSrRu3RpnzpwBANy4cQO9evWCtbU1zMzM0KRJE+zatavM64iiiK+++gpTpkxBnz590KxZM6xbtw63b9/G9u3btfpd0NMpzR4lhveyHgIAalmZoa27A/7JyMGepcMQ/3MoIhcNxWtNX5HOae5aB3VrW6JYLMahb0bg8k/j8dPc91UqLEbVqyHvseEjAHiYVwgTRXU0d7V/AXdGVLas7FwAgJWlqZo2D2FhZoxq1QzLPL5p1ymYGBuh9xstKiNEqgQlQ0IV3fSVzhKW9PR07N69G8HBwTAzMyt13MrK6qnnWlhYICIiApcuXcLixYuxatUqLFq0SDoeEBCAevXq4fTp0zh79iw+/fRTVK9eHQAQHByMvLw8HD58GHFxcfjyyy9hbm5e5nWSkpKQmpoKb29vaZ+lpSXatWuHmJiY57xz0oQgCJgb3A0n4m7g8vU7AID6dawBAJ8O7oy1O8/i7U/X4/zVFGyfHwjnujVU2wR2wfwfDmHAZxuQkf0QOxYNgZXFozH/6DMJeLWJA/q90RQGBgLq1LLAhMGdAAB2Ncv+nSCqbMXFxfhs4S9o19wZ7g3KTpzvZmRj/urdCOz7+lP7+eG3E3jbt7VK1YVkTtDSpqd0NiSUkJAAURTRuHFjjc+dMmWK9HP9+vUxfvx4bNq0CRMmTAAAJCcnIzQ0VOr78fkmycnJ6NevHzw8PAAAzs7OT71OamoqAMDW1lZlv62trXTsSXl5ecjLy5M+Z2VlaXJr9IT5n/jBzckG3UetlvYZGDz6PzIi8gw27o4FAMQl7Eanlk54v3srzPxuHwz+/18ZC344jB1HLgMAgsO34+LmcejbqQkiIs/gwJlETPtmLxaO7oWVk/yRl1+E+T8cwuvN6qO4WASRLoSG/4TL11Kw69vRZR7Pyn6Id8esRCMnO0z8sEeZbU5dSMJfSalYGTaoEiMlerF0lrCI4vP/hbB582YsWbIEiYmJyM7ORmFhIZRKpXR87NixCAoKwvr16+Ht7Y133nkHDRo0AACMGjUKH330Efbu3Qtvb2/069cPzZo1q/D9lJg7dy5mzJihtf6qsvBRPeD7mit6jF6N2//+l/il3r0PQHU+CgDEJ/+LejaWj9qkl26TX1CE6yn3pDYAsPznGCz/OQZ2NS2Qcf8hXrGzwvThb+J6yr1Kuy+ip5kwbwv2HP0TO7/5BHVtrUsdv5+Ti3c+WQELUwXWhw9H9acMB63/9Tg8XOuhhdsrZR4neeIqIfV0NiTk4uICQRA0nlgbExODgIAA9OjRA5GRkTh37hwmT56M/Pz/JkiGhYXh4sWL8PPzQ3R0NNzd3bFt2zYAQFBQEK5du4ZBgwYhLi4Obdq0wdKlS8u8lp2dHQAgLS1NZX9aWpp07EmTJk1CZmamtN28eVOj+6NHwkf1gF97N/QeF4Hk1AyVY8mpGbj9bxYaOtRS2d+wXk3cTHvU9vxfKcjNL1BpU83QAK/YWkltHpd69z5y8wvR7w0P3ErLwPmrKdq+JaKnEkURE+Ztwc6DF/Dr8pFwrFurVJus7IfoN3IZjKobYsOC/8FYUb3MvrIf5OHX/efwfu/XKjts0jLOYVFPZwlLjRo14Ovri2XLliEnJ6fU8YyMjDLPO378OBwdHTF58mS0adMGLi4uuHHjRql2rq6uGDNmDPbu3Qt/f3+sWbNGOubg4IARI0Zg69atGDduHFatWlXmtZycnGBnZ4f9+/dL+7KysnDy5El4enqWeY5CoYBSqVTZSDPzP/FDf+9mGD77Z2Q/yIeNtTlsrM1hbPRfQXDp5mP431vt0LujO5zsa+CzoW/A5ZVaWP//K4LuP8jDmh1n8OmQzujSpgEaOtTEgtE9AQDbD12U+hn5rhfcnWzQuH5tjH+/E0a/1x4Tv/6dQ0L0QoWGb8GW38/g21mBMDc1Rtq/WUj7NwsPcx/9Qywr+yH6jVqOB7n5WDJlIO5n50ptiopUnxm0LeoPFBYVo3/3trq4FaoAQdDOpq90uqx52bJl8PLywquvvoqZM2eiWbNmKCwsRFRUFFasWIHLly+XOsfFxQXJycnYtGkT2rZti507d0rVEwB4+PAhQkND8fbbb8PJyQm3bt3C6dOn0a9fPwDA6NGj0b17d7i6uuLevXs4cOAA3NzcyoxPEASMHj0as2fPhouLi7Ss2d7eng+xq0TD+rwKANj51Qcq+z/+cht+3BMLAFj5ywkYG1XD5x93g5WFCS5eS4V/6Dpcv/3fUM60lXtRWFSMlZ/6w1hRDWcv/40+4yOQ+f8rMADA+9WGGBfQAUbVq+HPxFQETP0R+04lVP5NEj1m9S+Pluf3GrFEZf/X0wIwsOdruBB/C2f/vA4AaO0/U6VN7PYwvGJfU/r8w28x6Nm5OSwtnr7CiOhlJIgVmUyiBSkpKZgzZw4iIyORkpKC2rVro3Xr1hgzZgw6d+78KMgnnnQ7YcIErF69Gnl5efDz88Nrr72GsLAwZGRkID8/H4GBgTh27BjS0tJQq1Yt+Pv7Y968eTA2NsbIkSPx+++/49atW1AqlejWrRsWLVqEmjVrlhlfyYPjvv32W2RkZKB9+/ZYvnw5XF1dy3V/WVlZsLS0hMJzIoRqCm18ZUSyk74/TNchEFWarKws2NWyQmZmZqVUzUv+nnAe+TMMFKVXzWqiOC8H15a+XWmx6pLOExZ9x4SFqgImLKTPXljCMupnGFYwYSnKy8G1JfqZsPDlh0RERCR7On+XEBEREXFZ87MwYSEiIpIBbazy0eN8hUNCREREJH+ssBAREcmAgYEgvXrkeYkVPF/OmLAQERHJAIeE1OOQEBEREckeKyxEREQywFVC6jFhISIikgEOCanHhIWIiEgGWGFRj3NYiIiISPZYYSEiIpIBVljUY4WFiIhIBkrmsFR008TcuXPRtm1bWFhYwMbGBn379kV8fLxKm9zcXAQHB6NmzZowNzdHv379kJaWptImOTkZfn5+MDU1hY2NDUJDQ1FYWKjS5uDBg2jVqhUUCgUaNmyIiIgIjWJlwkJERFRFHTp0CMHBwThx4gSioqJQUFAAHx8f5OTkSG3GjBmDHTt24KeffsKhQ4dw+/Zt+Pv7S8eLiorg5+eH/Px8HD9+HGvXrkVERASmTZsmtUlKSoKfnx+6dOmC2NhYjB49GkFBQdizZ0+5YxVEURS1c9tUlpLXhis8J0KoptB1OESVIn1/mK5DIKo0WVlZsKtlhczMTCiVykrp39LSEh6f/gZDY7MK9VWUm4O4L3o/d6z//PMPbGxscOjQIXTs2BGZmZmoXbs2Nm7ciLfffhsAcOXKFbi5uSEmJgavvfYafv/9d/Ts2RO3b9+Gra0tAGDlypWYOHEi/vnnHxgZGWHixInYuXMn/vzzT+laAwYMQEZGBnbv3l2u2FhhISIikgFtDgllZWWpbHl5eeWKITMzEwBQo0YNAMDZs2dRUFAAb29vqU3jxo3xyiuvICYmBgAQExMDDw8PKVkBAF9fX2RlZeHixYtSm8f7KGlT0kd5MGEhIiLSMw4ODrC0tJS2uXPnPvOc4uJijB49Gl5eXmjatCkAIDU1FUZGRrCyslJpa2tri9TUVKnN48lKyfGSY+raZGVl4eHDh+W6J64SIiIikgFtrhK6efOmypCQQvHsKQnBwcH4888/cfTo0QrFUFmYsBAREcmANp90q1QqNZrDEhISgsjISBw+fBj16tWT9tvZ2SE/Px8ZGRkqVZa0tDTY2dlJbU6dOqXSX8kqosfbPLmyKC0tDUqlEiYmJuWKkUNCREREVZQoiggJCcG2bdsQHR0NJycnleOtW7dG9erVsX//fmlffHw8kpOT4enpCQDw9PREXFwc7ty5I7WJioqCUqmEu7u71ObxPkralPRRHqywEBERyYAuHhwXHByMjRs34tdff4WFhYU058TS0hImJiawtLTEsGHDMHbsWNSoUQNKpRIjR46Ep6cnXnvtNQCAj48P3N3dMWjQIISHhyM1NRVTpkxBcHCwNBQ1YsQIfP3115gwYQI++OADREdHY8uWLdi5c2e5Y2XCQkREJAO6ePnhihUrAACdO3dW2b9mzRoMGTIEALBo0SIYGBigX79+yMvLg6+vL5YvXy61NTQ0RGRkJD766CN4enrCzMwMgYGBmDlzptTGyckJO3fuxJgxY7B48WLUq1cP3333HXx9fcsdKxMWIiIiGdBFhaU8j2IzNjbGsmXLsGzZsqe2cXR0xK5du9T207lzZ5w7d06j+B7HOSxEREQke6ywEBERyYEWhoSgv+8+ZMJCREQkB3xbs3ocEiIiIiLZY4WFiIhIBnSxSuhlwoSFiIhIBjgkpB6HhIiIiEj2WGEhIiKSAQ4JqceEhYiISAY4JKQeh4SIiIhI9lhhISIikgFWWNRjwkJERCQDnMOiHhMWIiIiGWCFRT3OYSEiIiLZY4WFiIhIBjgkpB4TFiIiIhngkJB6HBIiIiIi2WOFhYiISAYEaGFISCuRyBMTFiIiIhkwEAQYVDBjqej5csYhISIiIpI9VliIiIhkgKuE1GPCQkREJANcJaQeExYiIiIZMBAebRXtQ19xDgsRERHJHissREREciBoYUhHjyssTFiIiIhkgJNu1eOQEBEREckeKyxEREQyIPz/fxXtQ18xYSEiIpIBrhJSj0NCREREJHussBAREckAHxynXrkSlt9++63cHfbu3fu5gyEiIqqquEpIvXIlLH379i1XZ4IgoKioqCLxEBEREZVSroSluLi4suMgIiKq0gwEAQYVLJFU9Hw5q9AcltzcXBgbG2srFiIioiqLQ0LqabxKqKioCLNmzULdunVhbm6Oa9euAQCmTp2K77//XusBEhERVQUlk24ruukrjROWOXPmICIiAuHh4TAyMpL2N23aFN99951WgyMiIiICniNhWbduHb799lsEBATA0NBQ2t+8eXNcuXJFq8ERERFVFSVDQhXd9JXGc1j+/vtvNGzYsNT+4uJiFBQUaCUoIiKiqoaTbtXTuMLi7u6OI0eOlNr/888/o2XLlloJioiIiOhxGldYpk2bhsDAQPz9998oLi7G1q1bER8fj3Xr1iEyMrIyYiQiItJ7wv9vFe1DX2lcYenTpw927NiBffv2wczMDNOmTcPly5exY8cOvPnmm5URIxERkd7jKiH1nus5LB06dEBUVJS2YyEiIiIq03M/OO7MmTO4fPkygEfzWlq3bq21oIiIiKoaA+HRVtE+9JXGCcutW7fw3nvv4dixY7CysgIAZGRk4PXXX8emTZtQr149bcdIRESk9/i2ZvU0nsMSFBSEgoICXL58Genp6UhPT8fly5dRXFyMoKCgyoiRiIiIqjiNKyyHDh3C8ePH0ahRI2lfo0aNsHTpUnTo0EGrwREREVUlelwgqTCNExYHB4cyHxBXVFQEe3t7rQRFRERU1XBISD2Nh4TmzZuHkSNH4syZM9K+M2fO4JNPPsH8+fO1GhwREVFVUTLptqKbvipXhcXa2lola8vJyUG7du1Qrdqj0wsLC1GtWjV88MEH6Nu3b6UESkRERFVXuRKWr776qpLDICIiqto4JKReuRKWwMDAyo6DiIioSuOj+dV77gfHAUBubi7y8/NV9imVygoFRERERPQkjROWnJwcTJw4EVu2bMHdu3dLHS8qKtJKYERERFWJgSDAoIJDOhU9X840XiU0YcIEREdHY8WKFVAoFPjuu+8wY8YM2NvbY926dZURIxERkd4TBO1s+krjCsuOHTuwbt06dO7cGUOHDkWHDh3QsGFDODo6YsOGDQgICKiMOImIiKgK07jCkp6eDmdnZwCP5qukp6cDANq3b4/Dhw9rNzoiIqIqomSVUEU3faVxwuLs7IykpCQAQOPGjbFlyxYAjyovJS9DJCIiIs1wSEg9jROWoUOH4vz58wCATz/9FMuWLYOxsTHGjBmD0NBQrQdIREREpPEcljFjxkg/e3t748qVKzh79iwaNmyIZs2aaTU4IiKiqoKrhNTTuMLyJEdHR/j7+zNZISIiqgBdDAkdPnwYvXr1gr29PQRBwPbt21WODxkypNQcmW7duqm0SU9PR0BAAJRKJaysrDBs2DBkZ2ertLlw4QI6dOgAY2NjODg4IDw8XOPvp1wVliVLlpS7w1GjRmkcBBERUVWni0fz5+TkoHnz5vjggw/g7+9fZptu3bphzZo10meFQqFyPCAgACkpKYiKikJBQQGGDh2KDz/8EBs3bgQAZGVlwcfHB97e3li5ciXi4uLwwQcfwMrKCh9++GG5Yy1XwrJo0aJydSYIAhMWIiKil0T37t3RvXt3tW0UCgXs7OzKPHb58mXs3r0bp0+fRps2bQAAS5cuRY8ePTB//nzY29tjw4YNyM/Px+rVq2FkZIQmTZogNjYWCxcu1H7CUrIqiJ5fcuRnfG0B6S3rtiG6DoGo0ohF+c9upAUGqPg8jQrP8yjDwYMHYWNjA2tra7zxxhuYPXs2atasCQCIiYmBlZWVlKwAj+a3GhgY4OTJk3jrrbcQExODjh07wsjISGrj6+uLL7/8Evfu3YO1tXW54qjQu4SIiIhIO7Q5JJSVlaWyX6FQlBrKKY9u3brB398fTk5OSExMxGeffYbu3bsjJiYGhoaGSE1NhY2Njco51apVQ40aNZCamgoASE1NhZOTk0obW1tb6RgTFiIioirKwcFB5fP06dMRFhamcT8DBgyQfvbw8ECzZs3QoEEDHDx4EF27dq1omBphwkJERCQDggAYVHBVckmB5ubNmyrTEJ6nulIWZ2dn1KpVCwkJCejatSvs7Oxw584dlTaFhYVIT0+X5r3Y2dkhLS1NpU3J56fNjSlLZQx3ERERkYYMBO1swKNX5zy+aSthuXXrFu7evYs6deoAADw9PZGRkYGzZ89KbaKjo1FcXIx27dpJbQ4fPoyCggKpTVRUFBo1alTu4SCACQsREVGVlZ2djdjYWMTGxgJ4tMgmNjYWycnJyM7ORmhoKE6cOIHr169j//796NOnDxo2bAhfX18AgJubG7p164bhw4fj1KlTOHbsGEJCQjBgwADY29sDAAYOHAgjIyMMGzYMFy9exObNm7F48WKMHTtWo1ifK2E5cuQI3n//fXh6euLvv/8GAKxfvx5Hjx59nu6IiIiqPF28/PDMmTNo2bIlWrZsCQAYO3YsWrZsiWnTpsHQ0BAXLlxA79694erqimHDhqF169Y4cuSISsVmw4YNaNy4Mbp27YoePXqgffv2+Pbbb6XjlpaW2Lt3L5KSktC6dWuMGzcO06ZN02hJM/Acc1h++eUXDBo0CAEBATh37hzy8vIAAJmZmfj888+xa9cuTbskIiKq8gy0MIdF0/M7d+4MURSfenzPnj3P7KNGjRrSQ+KeplmzZjhy5IhmwT1B4wrL7NmzsXLlSqxatQrVq1eX9nt5eeGPP/6oUDBEREREZdG4whIfH4+OHTuW2m9paYmMjAxtxERERFTlPM+7gMrqQ19pXGGxs7NDQkJCqf1Hjx6Fs7OzVoIiIiKqakre1lzRTV9pnLAMHz4cn3zyCU6ePAlBEHD79m1s2LAB48ePx0cffVQZMRIREek9Ay1t+krjIaFPP/0UxcXF6Nq1Kx48eICOHTtCoVBg/PjxGDlyZGXESERERFWcxgmLIAiYPHkyQkNDkZCQgOzsbLi7u8Pc3Lwy4iMiIqoSOIdFved+NL+RkRHc3d21GQsREVGVZYCKz0ExgP5mLBonLF26dFH7YJro6OgKBURERET0JI0TlhYtWqh8LigoQGxsLP78808EBgZqKy4iIqIqhUNC6mmcsCxatKjM/WFhYcjOzq5wQERERFWRLp50+zLR2gqo999/H6tXr9ZWd0RERESS5550+6SYmBgYGxtrqzsiIqIqRRBQ4Um3HBJ6jL+/v8pnURSRkpKCM2fOYOrUqVoLjIiIqCrhHBb1NE5YLC0tVT4bGBigUaNGmDlzJnx8fLQWGBEREVEJjRKWoqIiDB06FB4eHrC2tq6smIiIiKocTrpVT6NJt4aGhvDx8eFbmYmIiLRM0NJ/+krjVUJNmzbFtWvXKiMWIiKiKqukwlLRTV9pnLDMnj0b48ePR2RkJFJSUpCVlaWyEREREWlbueewzJw5E+PGjUOPHj0AAL1791Z5RL8oihAEAUVFRdqPkoiISM9xDot65U5YZsyYgREjRuDAgQOVGQ8REVGVJAiC2nf1lbcPfVXuhEUURQBAp06dKi0YIiIiorJotKxZnzM3IiIiXeKQkHoaJSyurq7PTFrS09MrFBAREVFVxCfdqqdRwjJjxoxST7olIiIiqmwaJSwDBgyAjY1NZcVCRERUZRkIQoVffljR8+Ws3AkL568QERFVHs5hUa/cD44rWSVERERE9KKVu8JSXFxcmXEQERFVbVqYdKvHrxLSbA4LERERVQ4DCDCoYMZR0fPljAkLERGRDHBZs3oav/yQiIiI6EVjhYWIiEgGuEpIPSYsREREMsDnsKjHISEiIiKSPVZYiIiIZICTbtVjwkJERCQDBtDCkJAeL2vmkBARERHJHissREREMsAhIfWYsBAREcmAASo+7KHPwyb6fG9ERESkJ1hhISIikgFBECBUcEynoufLGRMWIiIiGRBQ8Zct62+6woSFiIhIFvikW/U4h4WIiIhkjxUWIiIimdDf+kjFMWEhIiKSAT6HRT0OCREREZHsscJCREQkA1zWrB4TFiIiIhngk27V0+d7IyIiIj3BCgsREZEMcEhIPSYsREREMsAn3arHISEiIiKSPVZYiIiIZIBDQuoxYSEiIpIBrhJSjwkLERGRDLDCop4+J2NERESkJ1hhISIikgGuElKPCQsREZEM8OWH6nFIiIiIiGSPFRYiIiIZMIAAgwoO6lT0fDljhYWIiEgGSoaEKrpp4vDhw+jVqxfs7e0hCAK2b9+uclwURUybNg116tSBiYkJvL29cfXqVZU26enpCAgIgFKphJWVFYYNG4bs7GyVNhcuXECHDh1gbGwMBwcHhIeHa/z9MGEhIiKqonJyctC8eXMsW7aszOPh4eFYsmQJVq5ciZMnT8LMzAy+vr7Izc2V2gQEBODixYuIiopCZGQkDh8+jA8//FA6npWVBR8fHzg6OuLs2bOYN28ewsLC8O2332oUK4eEiIiIZED4//8q2ocmunfvju7du5d5TBRFfPXVV5gyZQr69OkDAFi3bh1sbW2xfft2DBgwAJcvX8bu3btx+vRptGnTBgCwdOlS9OjRA/Pnz4e9vT02bNiA/Px8rF69GkZGRmjSpAliY2OxcOFClcTmWVhhISIikgFtDgllZWWpbHl5eRrHk5SUhNTUVHh7e0v7LC0t0a5dO8TExAAAYmJiYGVlJSUrAODt7Q0DAwOcPHlSatOxY0cYGRlJbXx9fREfH4979+6VOx4mLERERHrGwcEBlpaW0jZ37lyN+0hNTQUA2Nraquy3tbWVjqWmpsLGxkbleLVq1VCjRg2VNmX18fg1yoNDQkRERDIgaGGVUMmQ0M2bN6FUKqX9CoWiQv3KASssREREMqDNISGlUqmyPU/CYmdnBwBIS0tT2Z+WliYds7Ozw507d1SOFxYWIj09XaVNWX08fo3yYMJCREQkA7pY1qyOk5MT7OzssH//fmlfVlYWTp48CU9PTwCAp6cnMjIycPbsWalNdHQ0iouL0a5dO6nN4cOHUVBQILWJiopCo0aNYG1tXe54mLAQERFVUdnZ2YiNjUVsbCyARxNtY2NjkZycDEEQMHr0aMyePRu//fYb4uLiMHjwYNjb26Nv374AADc3N3Tr1g3Dhw/HqVOncOzYMYSEhGDAgAGwt7cHAAwcOBBGRkYYNmwYLl68iM2bN2Px4sUYO3asRrFyDgsREZEM6GJZ85kzZ9ClSxfpc0kSERgYiIiICEyYMAE5OTn48MMPkZGRgfbt22P37t0wNjaWztmwYQNCQkLQtWtXGBgYoF+/fliyZIl03NLSEnv37kVwcDBat26NWrVqYdq0aRotaQYAQRRFUaMzSCNZWVmwtLRE2t1MlQlQRPrEum2IrkMgqjRiUT7y4lYhM7Ny/hwv+Xvi19PXYGZuUaG+crLvo09b50qLVZc4JERERESyxyEhIiIiGdDFkNDLhAkLERGRDGhjlY82VwnJDYeEiIiISPZYYSEiIpIBARUf0tHjAgsTFiIiIjkwEB5tFe1DX3FIiIiIiGTvpaiwCIKAbdu2SU/Wo6pl4Zo9iDxwHldvpMFYUR2vNnNGWEgfuNT/7+2fEVuP4uc9Z3Ah/hbu5+TienQ4LC1MVfpp1nsabqakq+ybFtwbY4b4vJD7ICoxZogPenZpDhdHW+TmFeDUhWsI+/pXJNz4750sNjUtMHPUW+jcrjHMTRVIuHEHC1bvwY4DsVIbK6UpwkPfgW/7phBFEb9Fx2LSgp+R8zBfatPXuyXGDvVFg1dscPdeNlZtOYSlP+wHyQ9XCamn8wpLamoqRo4cCWdnZygUCjg4OKBXr14q7y7Qpa1bt8LHxwc1a9aEIAjS44vpxTn+RwKC3umIvavHY+vXISgoLIL/yK+R8zBPavMwtwBdPd2fmXx89j8/XPn9c2n78N1OlR0+USmvt2qI7346DJ8P5sM/5GtUr2aIrUtDYGpsJLVZETYYDR1tMHDsN/B673PsOBCLNXM/gIdrPanNqlmBaOxcB/4hX2PAmJV4vWVDfPXZQOm49+vu+HbWEKz55SheHzAH47/cjI8GvoHh73R8ofdL5SO3dwnJjU4rLNevX4eXlxesrKwwb948eHh4oKCgAHv27EFwcDCuXLmiy/AAADk5OWjfvj369++P4cOH6zqcKunnpcEqn5dPfx8uPpMQe/kmvFo1BAB8NPDRo6WPnv1LbV/mpsawraVfT3+kl887o5arfP54xg9IiPoCLdwccPxcIgDg1WbOGP/FJvxx6QYAYMHqPfj4vTfQws0BcX/dgmt9W3i/3gRdBocj9nIyAGDi/J+w5auPMHXxNqT+m4l3u7+KnQfPY83WowCAG3/fxaKIvfgk8E2s+unwC7xjKg8BFZ80q8f5im4rLB9//DEEQcCpU6fQr18/uLq6okmTJhg7dixOnDjx1PMmTpwIV1dXmJqawtnZGVOnTlV5C+T58+fRpUsXWFhYQKlUonXr1jhz5gwA4MaNG+jVqxesra1hZmaGJk2aYNeuXU+91qBBgzBt2jR4e3tr78apQrKycwEA1krTZ7Qs7au1e+HsPQEdA77AkvX7UFhYpO3wiDSmNH/0XpZ7WQ+kfacuXMNbb7aGldIUgiDA/83WUCiq4ejZqwCAth5OyMh6ICUrAHDwVDyKi0W0buoIADAyqoa8/EKVa+Xm5aOurTUc6tSo7Nsi0iqdVVjS09Oxe/duzJkzB2ZmZqWOW1lZPfVcCwsLREREwN7eHnFxcRg+fDgsLCwwYcIEAEBAQABatmyJFStWwNDQELGxsahevToAIDg4GPn5+Th8+DDMzMxw6dIlmJuba+2+8vLykJf331BFVlaW1vomoLi4GJMW/ox2zZ3h3tBeo3P/924nNG/sACulGU5duIaZy35D2r+ZmDOmXyVFS/RsgiBg7ti3cSI2EZcTU6T9QyetxurPP0DS/nAUFBbhYW4+BoWuQtKtfwEAtjWV+OfefZW+ioqKcS/rAWxrPqoiRp+4jDlj/NEx0hVHzlyFs0NtBAd0BQDY1bIsNaeLdMsAAgwqOKZjoMc1Fp0lLAkJCRBFEY0bN9b43ClTpkg/169fH+PHj8emTZukhCU5ORmhoaFS3y4uLlL75ORk9OvXDx4eHgAAZ2fnitxGKXPnzsWMGTO02if9Z3z4FlxOTMHvq8ZofG7JH9QA0NSlLoyqV8OYz3/EtODeUBhV12aYROU2f0J/uDWog+7DF6nsnzyiJywtTNDn4yVIz8hBj07NsGbuB+gx/CtcSrxdrr7XbjsGp7q1sGnhCFSvZoj7OblYuekgJv3PD8XFxZVxO1QBHBJST2dDQhV5SfTmzZvh5eUFOzs7mJubY8qUKUhO/q8sOnbsWAQFBcHb2xtffPEFEhMTpWOjRo3C7Nmz4eXlhenTp+PChQsVuo8nTZo0CZmZmdJ28+ZNrfZflYWGb8GeI39ix4pRqGtrXeH+Wjepj8KiYiTf5r8ySTfCQ9+Bb4em6PXREty+kyHtr1+3Fj58txNGzvoBh0//hT+v/o3w737HucvJCPr/CbNpd7NQ21r1zb6GhgawVpoi7e5/ld2wr39FvU7j0Kz3NDTq9pk0J+b633cr/waJtEhnCYuLiwsEQdB4Ym1MTAwCAgLQo0cPREZG4ty5c5g8eTLy8/9bxhcWFoaLFy/Cz88P0dHRcHd3x7Zt2wAAQUFBuHbtGgYNGoS4uDi0adMGS5cu1dp9KRQKKJVKlY0qRhRFhIZvwc6D5/HbilFwrFtLK/3G/XULBgYCateo2OvciZ5HeOg78OvcHL0/WoLk26rJQ8lqoeJi1X/YFRWJEP7/yWCn45JgpTRF88YO0vGObVxhYCDg7J83VM4rLhaR8k8mCgqL0M+nNU5duIa7GdmVcVtUEYKWNj2ls4SlRo0a8PX1xbJly5CTk1PqeEZGRpnnHT9+HI6Ojpg8eTLatGkDFxcX3Lhxo1Q7V1dXjBkzBnv37oW/vz/WrFkjHXNwcMCIESOwdetWjBs3DqtWrdLafZH2jf9yC7b8fhqrZg2Buakx0v7NQtq/WXiY+1+SmvZvFuLib+HazUfj+xcTbiMu/hbuZT763Tp14RpWbDyAuL9u4fqtf7Hl99OYvOgX9O/eFlbPMXmXqCLmT+yP/t3bYvjUCGQ/yIVNTQvY1LSAseLR0ORf11ORmHwHiya9h1bujqhftxaCA95Al3aNsOvg+f9vk4Z9xy9i8eSBaOXuiHbNnBEe2h9b9/6B1H8zAQA1LM0w1L89XBxt0dS1LuaO64c+XVti0oJfdHbv9HSClv7TVzpd1rxs2TJ4eXnh1VdfxcyZM9GsWTMUFhYiKioKK1aswOXLl0ud4+LiguTkZGzatAlt27bFzp07peoJADx8+BChoaF4++234eTkhFu3buH06dPo1+/RxMrRo0eje/fucHV1xb1793DgwAG4ubk9Ncb09HQkJyfj9u1HY8bx8fEAADs7O9jZ2Wnz66CnWP3LEQBAzxGLVfYvm/Y+BvZ6DQCwZusRfLnqd+mY34dfqbRRGFXH1qiz+GLVLuQXFMLRviY+eq8LggPeeDE3QfSYYW8/GtbZ+c1olf0fz1iPHyNPorCoGP1Hr8D0kD74ceH/YGaqQNLNf/Bx2HpEHb8ktR8+dS3mhfbH9uUjpQfHfTr/J5U+B/i1w8xP3oIgPKrK9BqxWBoWInqZCGJFJpNoQUpKCubMmYPIyEikpKSgdu3aaN26NcaMGYPOnTs/CvKJJ91OmDABq1evRl5eHvz8/PDaa68hLCwMGRkZyM/PR2BgII4dO4a0tDTUqlUL/v7+mDdvHoyNjTFy5Ej8/vvvuHXrFpRKJbp164ZFixahZs2aZcYXERGBoUOHlto/ffp0hIWFPfP+srKyYGlpibS7mRweIr1l3TZE1yEQVRqxKB95cauQmVk5f46X/D2xPzYZ5hYV6z/7fha6tnil0mLVJZ0nLPqOCQtVBUxYSJ+9qIQlWksJyxt6mrDo/NH8RERERM/yUrz8kIiISO/xQSxqMWEhIiKSAb6tWT0mLERERDKgjbct6/PbmjmHhYiIiGSPFRYiIiIZ4BQW9ZiwEBERyQEzFrU4JERERESyxwoLERGRDHCVkHpMWIiIiGSAq4TU45AQERERyR4rLERERDLAObfqMWEhIiKSA2YsanFIiIiIiGSPFRYiIiIZ4Coh9ZiwEBERyQBXCanHhIWIiEgGOIVFPc5hISIiItljhYWIiEgOWGJRiwkLERGRDHDSrXocEiIiIiLZY4WFiIhIBrhKSD0mLERERDLAKSzqcUiIiIiIZI8VFiIiIjlgiUUtJixEREQywFVC6nFIiIiIiGSPFRYiIiIZ4Coh9ZiwEBERyQCnsKjHhIWIiEgOmLGoxTksREREJHussBAREckAVwmpx4SFiIhIDrQw6VaP8xUOCREREZH8scJCREQkA5xzqx4TFiIiIjlgxqIWh4SIiIhI9lhhISIikgGuElKPCQsREZEM8NH86nFIiIiIiGSPFRYiIiIZ4Jxb9VhhISIikgNBS5sGwsLCIAiCyta4cWPpeG5uLoKDg1GzZk2Ym5ujX79+SEtLU+kjOTkZfn5+MDU1hY2NDUJDQ1FYWPgcX4B6rLAQERHJgK4m3TZp0gT79u2TPler9l9qMGbMGOzcuRM//fQTLC0tERISAn9/fxw7dgwAUFRUBD8/P9jZ2eH48eNISUnB4MGDUb16dXz++ecVupcnMWEhIiKqwqpVqwY7O7tS+zMzM/H9999j48aNeOONNwAAa9asgZubG06cOIHXXnsNe/fuxaVLl7Bv3z7Y2tqiRYsWmDVrFiZOnIiwsDAYGRlpLU4OCREREcmAgP9WCj339v99ZWVlqWx5eXlPve7Vq1dhb28PZ2dnBAQEIDk5GQBw9uxZFBQUwNvbW2rbuHFjvPLKK4iJiQEAxMTEwMPDA7a2tlIbX19fZGVl4eLFi1r9fpiwEBERyYA2p7A4ODjA0tJS2ubOnVvmNdu1a4eIiAjs3r0bK1asQFJSEjp06ID79+8jNTUVRkZGsLKyUjnH1tYWqampAIDU1FSVZKXkeMkxbeKQEBERkZ65efMmlEql9FmhUJTZrnv37tLPzZo1Q7t27eDo6IgtW7bAxMSk0uPUBCssREREMlDh4aDHHjynVCpVtqclLE+ysrKCq6srEhISYGdnh/z8fGRkZKi0SUtLk+a82NnZlVo1VPK5rHkxFcGEhYiISBZ0sK75CdnZ2UhMTESdOnXQunVrVK9eHfv375eOx8fHIzk5GZ6engAAT09PxMXF4c6dO1KbqKgoKJVKuLu7VyiWJ3FIiIiIqIoaP348evXqBUdHR9y+fRvTp0+HoaEh3nvvPVhaWmLYsGEYO3YsatSoAaVSiZEjR8LT0xOvvfYaAMDHxwfu7u4YNGgQwsPDkZqaiilTpiA4OLjcVZ3yYsJCREQkA7p4l9CtW7fw3nvv4e7du6hduzbat2+PEydOoHbt2gCARYsWwcDAAP369UNeXh58fX2xfPly6XxDQ0NERkbio48+gqenJ8zMzBAYGIiZM2dW7EbKIIiiKGq9V5JkZWXB0tISaXczVSZAEekT67Yhug6BqNKIRfnIi1uFzMzK+XO85O+JKzf+gUUF+7+flYXGjrUrLVZd4hwWIiIikj0OCREREcmALoaEXiZMWIiIiGRAV+8SelkwYSEiIpKDiq9Krvj5MsY5LERERCR7rLAQERHJAAss6jFhISIikgFOulWPQ0JEREQke6ywEBERyQBXCanHhIWIiEgOOIlFLQ4JERERkeyxwkJERCQDLLCox4SFiIhIBrhKSD0OCREREZHsscJCREQkCxVfJaTPg0JMWIiIiGSAQ0LqcUiIiIiIZI8JCxEREckeh4SIiIhkgENC6jFhISIikgE+ml89DgkRERGR7LHCQkREJAMcElKPCQsREZEM8NH86nFIiIiIiGSPFRYiIiI5YIlFLSYsREREMsBVQupxSIiIiIhkjxUWIiIiGeAqIfWYsBAREckAp7Cox4SFiIhIDpixqMU5LERERCR7rLAQERHJAFcJqceEhYiISAY46VY9JiyVTBRFAMD9rCwdR0JUecSifF2HQFRpSn6/S/48ryxZWvh7Qht9yBUTlkp2//59AEBDJwcdR0JERBVx//59WFpaar1fIyMj2NnZwUVLf0/Y2dnByMhIK33JiSBWdspYxRUXF+P27duwsLCAoM+1OpnIysqCg4MDbt68CaVSqetwiLSOv+MvniiKuH//Puzt7WFgUDlrVXJzc5Gfr51KpZGREYyNjbXSl5ywwlLJDAwMUK9ePV2HUeUolUr+YU56jb/jL1ZlVFYeZ2xsrJdJhjZxWTMRERHJHhMWIiIikj0mLKRXFAoFpk+fDoVCoetQiCoFf8epquKkWyIiIpI9VliIiIhI9piwEBERkewxYSEiIiLZY8JCsiYIArZv367rMIgqBX+/icqPCQvpTGpqKkaOHAlnZ2coFAo4ODigV69e2L9/v65DA/Do6ZbTpk1DnTp1YGJiAm9vb1y9elXXYdFLQu6/31u3boWPjw9q1qwJQRAQGxur65CI1GLCQjpx/fp1tG7dGtHR0Zg3bx7i4uKwe/dudOnSBcHBwboODwAQHh6OJUuWYOXKlTh58iTMzMzg6+uL3NxcXYdGMvcy/H7n5OSgffv2+PLLL3UdClH5iEQ60L17d7Fu3bpidnZ2qWP37t2TfgYgbtu2Tfo8YcIE0cXFRTQxMRGdnJzEKVOmiPn5+dLx2NhYsXPnzqK5ubloYWEhtmrVSjx9+rQoiqJ4/fp1sWfPnqKVlZVoamoquru7izt37iwzvuLiYtHOzk6cN2+etC8jI0NUKBTijz/+WMG7J30n99/vxyUlJYkAxHPnzj33/RK9CHyXEL1w6enp2L17N+bMmQMzM7NSx62srJ56roWFBSIiImBvb4+4uDgMHz4cFhYWmDBhAgAgICAALVu2xIoVK2BoaIjY2FhUr14dABAcHIz8/HwcPnwYZmZmuHTpEszNzcu8TlJSElJTU+Ht7S3ts7S0RLt27RATE4MBAwZU4BsgffYy/H4TvYyYsNALl5CQAFEU0bhxY43PnTJlivRz/fr1MX78eGzatEn6Az05ORmhoaFS3y4uLlL75ORk9OvXDx4eHgAAZ2fnp14nNTUVAGBra6uy39bWVjpGVJaX4feb6GXEOSz0wokVeLjy5s2b4eXlBTs7O5ibm2PKlClITk6Wjo8dOxZBQUHw9vbGF198gcTEROnYqFGjMHv2bHh5eWH69Om4cOFChe6DqCz8/SaqHExY6IVzcXGBIAi4cuWKRufFxMQgICAAPXr0QGRkJM6dO4fJkycjPz9fahMWFoaLFy/Cz88P0dHRcHd3x7Zt2wAAQUFBuHbtGgYNGoS4uDi0adMGS5cuLfNadnZ2AIC0tDSV/WlpadIxorK8DL/fRC8l3U6hoaqqW7duGk9KnD9/vujs7KzSdtiwYaKlpeVTrzNgwACxV69eZR779NNPRQ8PjzKPlUy6nT9/vrQvMzOTk26pXOT++/04TrqllwUrLKQTy5YtQ1FREV599VX88ssvuHr1Ki5fvowlS5bA09OzzHNcXFyQnJyMTZs2ITExEUuWLJH+dQkADx8+REhICA4ePIgbN27g2LFjOH36NNzc3AAAo0ePxp49e5CUlIQ//vgDBw4ckI49SRAEjB49GrNnz8Zvv/2GuLg4DB48GPb29ujbt6/Wvw/SL3L//QYeTQ6OjY3FpUuXAADx8fGIjY3lHC2SL11nTFR13b59WwwODhYdHR1FIyMjsW7dumLv3r3FAwcOSG3wxLLP0NBQsWbNmqK5ubn47rvviosWLZL+BZqXlycOGDBAdHBwEI2MjER7e3sxJCREfPjwoSiKohgSEiI2aNBAVCgUYu3atcVBgwaJ//7771PjKy4uFqdOnSra2tqKCoVC7Nq1qxgfH18ZXwXpIbn/fq9Zs0YEUGqbPn16JXwbRBUniGIFZogRERERvQAcEiIiIiLZY8JCREREsseEhYiIiGSPCQsRERHJHhMWIiIikj0mLERERCR7TFiIiIhI9piwEFUBQ4YMUXlCb+fOnTF69OgXHsfBgwchCAIyMjKe2kYQBGzfvr3cfYaFhaFFixYViuv69esQBAGxsbEV6oeIKg8TFiIdGTJkCARBgCAIMDIyQsOGDTFz5kwUFhZW+rW3bt2KWbNmlatteZIMIqLKVk3XARBVZd26dcOaNWuQl5eHXbt2ITg4GNWrV8ekSZNKtc3Pz4eRkZFWrlujRg2t9ENE9KKwwkKkQwqFAnZ2dnB0dMRHH30Eb29v/PbbbwD+G8aZM2cO7O3t0ahRIwDAzZs30b9/f1hZWaFGjRro06cPrl+/LvVZVFSEsWPHwsrKCjVr1sSECRPw5Bs4nhwSysvLw8SJE+Hg4ACFQoGGDRvi+++/x/Xr19GlSxcAgLW1NQRBwJAhQwAAxcXFmDt3LpycnGBiYoLmzZvj559/VrnOrl274OrqChMTE3Tp0kUlzvKaOHEiXF1dYWpqCmdnZ0ydOhUFBQWl2n3zzTdwcHCAqakp+vfvj8zMTJXj3333Hdzc3GBsbIzGjRtj+fLlGsdCRLrDhIVIRkxMTJCfny993r9/P+Lj4xEVFYXIyEgUFBTA19cXFhYWOHLkCI4dOwZzc3N069ZNOm/BggWIiIjA6tWrcfToUaSnp6u89bcsgwcPxo8//oglS5bg8uXL+Oabb2Bubg4HBwf88ssvAB69zTclJQWLFy8GAMydOxfr1q3DypUrcfHiRYwZMwbvv/8+Dh06BOBRYuXv749evXohNjYWQUFB+PTTTzX+TiwsLBAREYFLly5h8eLFWLVqFRYtWqTSJiEhAVu2bMGOHTuwe/dunDt3Dh9//LF0fMOGDZg2bRrmzJmDy5cv4/PPP8fUqVOxdu1ajeMhIh3R8csXiaqswMBAsU+fPqIoPnozdFRUlKhQKMTx48dLx21tbcW8vDzpnPXr14uNGjUSi4uLpX15eXmiiYmJuGfPHlEURbFOnTpieHi4dLygoECsV6+edC1RFMVOnTqJn3zyiSiKohgfHy8CEKOiosqM88CBAyIA8d69e9K+3Nxc0dTUVDx+/LhK22HDhonvvfeeKIqiOGnSJNHd3V3l+MSJE0v19SQ88QbjJ82bN09s3bq19Hn69OmioaGheOvWLWnf77//LhoYGIgpKSmiKIpigwYNxI0bN6r0M2vWLNHT01MURVFMSkoSAYjnzp176nWJSLc4h4VIhyIjI2Fubo6CggIUFxdj4MCBCAsLk457eHiozFs5f/48EhISYGFhodJPbm4uEhMTkZmZiZSUFLRr1046Vq1aNbRp06bUsFCJ2NhYGBoaolOnTuWOOyEhAQ8ePMCbb76psj8/Px8tW7YEAFy+fFklDgDw9PQs9zVKbN68GUuWLEFiYiKys7NRWFgIpVKp0uaVV15B3bp1Va5TXFyM+Ph4WFhYIDExEcOGDcPw4cOlNoWFhbC0tNQ4HiLSDSYsRDrUpUsXrFixAkZGRrC3t0e1aqr/S5qZmal8zs7ORuvWrbFhw4ZSfdWuXfu5YjAxMdH4nOzsbADAzp07VRIF4NG8HG2JiYlBQEAAZsyYAV9fX1haWmLTpk1YsGCBxrGuWrWqVAJlaGiotViJqHIxYSHSITMzMzRs2LDc7Vu1aoXNmzfDxsamVJWhRJ06dXDy5El07NgRwKNKwtmzZ9GqVasy23t4eKC4uBiHDh2Ct7d3qeMlFZ6ioiJpn7u7OxQKBZKTk59amXFzc5MmEJc4ceLEs2/yMcePH4ejoyMmT54s7btx40apdsnJybh9+zbs7e2l6xgYGKBRo0awtbWFvb09rl27hoCAAI2uT0TywUm3RC+RgIAA1KpVC3369MGRI0eQlJSEgwcPYtSoUbh16xYA4JNPPsEXX3yB7du348qVK/j444/VPkOlfv36CAwMxAcffIDt27dLfW7ZsgUA4OjoCEEQEBkZiX/++QfZ2dmwsLDA+PHjMWbMGKxduxaJiYn4448/sHTpUmki64gRI3D16lWEhoYiPj4eGzduREREhEb36+LiguTkZGzatAmJiYlYsmRJmROIjY2NERgYiPPnz+PIkSMYNWoU+vfvDzs7OwDAjBkzMHfuXCxZsgR//fUX4uLisGbNGixcuFCjeIhId5iwEL1ETE1NcfjwYbzyyivw9/eHm5sbhg0bhtzcXKniMm7cOAwaNAiBgYHw9PSEhYUF3nrrLbX9rlixAm+//TY+/vhjNG7cGMOHD0dOTg4AoG7dupgxYwY+/fRT2NraIiQkBAAwa9YsTJ06FXPnzoWbmxu6deuGnTt3wsnJCcCjeSW//PILtm/fjubNm2PlypX4/PPPNbrf3r17Y8yYMQgJCUGLFi1w/PhxTJ06tVS7hg0bwt/fHz169ICPjw+aNWumsmw5KCgI3333HdasWQMPDw906tQJERERUqxEJH+C+LSZeEREREQywQoLERERyR4TFiIiIpI9JixEREQke0xYiIiISPaYsBAREZHsMWEhIiIi2WPCQkRERLLHhIWIiIhkjwkLERERyR4TFiIiIpI9JixEREQke0xYiIiISPb+D8eEamDpGfNzAAAAAElFTkSuQmCC",
|
||
"text/plain": [
|
||
"<Figure size 640x480 with 2 Axes>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay\n",
|
||
"import matplotlib.pyplot as plt\n",
|
||
"\n",
|
||
"# 假设 predictions.label_ids 是真实的标签，preds 是模型的预测\n",
|
||
"cm = confusion_matrix(predictions.label_ids, preds)\n",
|
||
"\n",
|
||
"# 可视化混淆矩阵\n",
|
||
"disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['Class 0', 'Class 1'])\n",
|
||
"disp.plot(cmap=plt.cm.Blues)\n",
|
||
"plt.title('Confusion Matrix')\n",
|
||
"plt.show()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "23e3a640-88d7-4a1e-8515-7c417d50f018",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": []
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.12.3"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|