daviddoji/vanilla-machine-learning
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update seq2seq training

Browse Source
This commit is contained in:
ritchie46
2019-02-20 11:40:27 +01:00
parent 34a55eeadb
commit 47b5045ca9
5 changed files with 258 additions and 216 deletions
Download Patch File Download Diff File Expand all files Collapse all files

466
seq2seq/attention_decoder.ipynb
View File

@@ -2,7 +2,7 @@
"cells": [ "cells": [
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 1, "execution_count": 2,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
@@ -20,15 +20,29 @@
"from torch import nn\n", "from torch import nn\n",
"import re\n", "import re\n",
"import os\n", "import os\n",
"\n",
"try:\n",
" from tensorboardX import SummaryWriter\n", " from tensorboardX import SummaryWriter\n",
"except ModuleNotFoundError:\n",
" pass\n",
"\n", "\n",
"device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
"print(device)" "print(device)"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Sequence to sequence learning\n",
"We will use pytorch to translate short sentences from French to English and vice versa\n",
"\n",
"![](img/hello-lead.png)"
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 2, "execution_count": 3,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
@@ -39,7 +53,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 3, "execution_count": 4,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
@@ -66,14 +80,45 @@
" print(f.read(200))" " print(f.read(200))"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Preparing the data I\n",
"\n",
"* Create a Language class that maps indexes to words and words to indexes\n",
"\n",
"**indexes to word**\n",
"```python\n",
"{0: SOS,\n",
" 1: EOS,\n",
" 2: The\n",
" ...\n",
" n: World\n",
"}\n",
"```\n",
"\n",
"**words to indexes**\n",
"```python\n",
"{SOS: 0,\n",
" EOS: 1,\n",
" The: 2\n",
" ...\n",
" World: n\n",
"}\n",
"```\n",
"\n",
"* implement functions to convert the letters to asscii and remove rare letters. (á, ò, ê -> a, o, e)"
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 5,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"\n", "\n",
"class Lang:\n", "class Language:\n",
" \"\"\"\n", " \"\"\"\n",
" Utility class that serves as a language dictionary\n", " Utility class that serves as a language dictionary\n",
" \"\"\"\n", " \"\"\"\n",
@@ -142,27 +187,31 @@
" # Reverse pairs, make Lang instances\n", " # Reverse pairs, make Lang instances\n",
" if reverse:\n", " if reverse:\n",
" pairs = [list(reversed(p)) for p in pairs]\n", " pairs = [list(reversed(p)) for p in pairs]\n",
" input_lang = Lang(lang2)\n", " input_lang = Language(lang2)\n",
" output_lang = Lang(lang1)\n", " output_lang = Language(lang1)\n",
" else:\n", " else:\n",
" input_lang = Lang(lang1)\n", " input_lang = Language(lang1)\n",
" output_lang = Lang(lang2)\n", " output_lang = Language(lang2)\n",
"\n", "\n",
" return input_lang, output_lang, pairs" " return input_lang, output_lang, pairs"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Preparing the data II\n",
"Since there are a lot of example sentences and we want to train something quickly, we'll trim the data set to only relatively short and simple sentences. \n",
"Here the maximum length is 10 words (that includes ending punctuation) and we're filtering to sentences that translate to the form \"I am\" or \"He is\" etc. \n",
"(accounting for apostrophes replaced earlier).\n"
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": 6,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"# Since there are a lot of example sentences and we want to train something quickly, we'll trim the data set to only relatively short and simple sentences. \n",
"# Here the maximum length is 10 words (that includes ending punctuation) and we're filtering to sentences that translate to the form \"I am\" or \"He is\" etc. \n",
"# (accounting for apostrophes replaced earlier).\n",
"\n",
"\n",
"\n",
"def filter_pairs(pairs):\n", "def filter_pairs(pairs):\n",
" MAX_LENGTH = 10\n", " MAX_LENGTH = 10\n",
" \n", " \n",
@@ -183,37 +232,17 @@
] ]
}, },
{ {
"cell_type": "code", "cell_type": "markdown",
"execution_count": 6,
"metadata": {}, "metadata": {},
"outputs": [],
"source": [ "source": [
"class Data:\n", "# Preparing the data III\n",
" def __init__(self, pairs, lang_1, lang_2):\n",
" self.pairs = np.array(pairs)\n",
" np.random.seed(9)\n",
" np.random.shuffle(self.pairs)\n",
" idx_1 = [[lang_1.word2index[word] for word in s.split(' ')] \n",
" for s in self.pairs[:, 0]]\n",
" idx_2 = [[lang_2.word2index[word] for word in s.split(' ')]\n",
" for s in self.pairs[:, 1]]\n",
" self.idx_pairs = np.array(list(zip(idx_1, idx_2)))\n",
" self.shuffle_idx = np.arange(len(pairs))\n",
"\n", "\n",
" def __str__(self):\n", "Read the data from the text files, normalize the sentences, create the Language instances from the Language class and wrap the two languages in a Data class so we can shuffle the sentences and query them later."
" return(self.pairs)\n",
" \n",
" def shuffle(self):\n",
" np.random.shuffle(self.shuffle_idx)\n",
" self.pairs = self.pairs[self.shuffle_idx]\n",
" self.idx_pairs = self.idx_pairs[self.shuffle_idx] \n",
" \n",
" "
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 7, "execution_count": 10,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
@@ -235,14 +264,34 @@
"array(['we are even EOS', 'nous sommes a egalite EOS'], dtype='<U60')" "array(['we are even EOS', 'nous sommes a egalite EOS'], dtype='<U60')"
] ]
}, },
"execution_count": 7, "execution_count": 10,
"metadata": {}, "metadata": {},
"output_type": "execute_result" "output_type": "execute_result"
} }
], ],
"source": [ "source": [
"class Data:\n",
" def __init__(self, pairs, lang_1, lang_2):\n",
" self.pairs = np.array(pairs)\n",
" np.random.seed(9)\n",
" np.random.shuffle(self.pairs)\n",
" idx_1 = [[lang_1.word2index[word] for word in s.split(' ')] \n",
" for s in self.pairs[:, 0]]\n",
" idx_2 = [[lang_2.word2index[word] for word in s.split(' ')]\n",
" for s in self.pairs[:, 1]]\n",
" self.idx_pairs = np.array(list(zip(idx_1, idx_2)))\n",
" self.shuffle_idx = np.arange(len(pairs))\n",
" \n",
" def __str__(self):\n",
" return(self.pairs)\n",
" \n",
" def shuffle(self):\n",
" np.random.shuffle(self.shuffle_idx)\n",
" self.pairs = self.pairs[self.shuffle_idx]\n",
" self.idx_pairs = self.idx_pairs[self.shuffle_idx] \n",
" \n",
"def prepare_data(lang1, lang2, reverse=False):\n", "def prepare_data(lang1, lang2, reverse=False):\n",
" # read_langs initialized the Lang objects (still empty) and returns the pair sentences.\n", " # read_langs initialized the Language objects (still empty) and returns the pair sentences.\n",
" input_lang, output_lang, pairs = read_langs(lang1, lang2, reverse)\n", " input_lang, output_lang, pairs = read_langs(lang1, lang2, reverse)\n",
" print(\"Read %s sentence pairs\" % len(pairs))\n", " print(\"Read %s sentence pairs\" % len(pairs))\n",
" \n", " \n",
@@ -269,7 +318,11 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"The Encoder\n", "## Sequence to sequence model\n",
"\n",
"![](img/seq2seq.png)\n",
"\n",
"## The Encoder\n",
"\n", "\n",
"The encoder of a seq2seq network is a RNN that outputs some value for every word from the input sentence. For every input word the encoder outputs a vector and a hidden state, and uses the hidden state for the next input word.\n", "The encoder of a seq2seq network is a RNN that outputs some value for every word from the input sentence. For every input word the encoder outputs a vector and a hidden state, and uses the hidden state for the next input word.\n",
"\n", "\n",
@@ -282,7 +335,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 8, "execution_count": 12,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
@@ -291,7 +344,7 @@
"torch.Size([5, 1, 2])" "torch.Size([5, 1, 2])"
] ]
}, },
"execution_count": 8, "execution_count": 12,
"metadata": {}, "metadata": {},
"output_type": "execute_result" "output_type": "execute_result"
} }
@@ -324,9 +377,9 @@
" return x, h\n", " return x, h\n",
" \n", " \n",
"\n", "\n",
"m = Encoder(eng.n_words, 10, 2, False, 'cpu')\n", "m = Encoder(eng.n_words, 10, 2, False, device)\n",
"scentence = torch.tensor([400, 1, 2, 6, 8])\n", "sentence = torch.tensor([400, 1, 2, 6, 8], device=device)\n",
"a = m(scentence)\n", "a = m(sentence)\n",
"a[0].shape" "a[0].shape"
] ]
}, },
@@ -340,22 +393,22 @@
"\n", "\n",
"At every step of decoding, the decoder is given an input token and hidden state. The initial input token is the start-of-string <SOS> token, and the first hidden state is the context vector (the encoder’s last hidden state).\n", "At every step of decoding, the decoder is given an input token and hidden state. The initial input token is the start-of-string <SOS> token, and the first hidden state is the context vector (the encoder’s last hidden state).\n",
" \n", " \n",
"![](img/decoder-network.png)\n", "![](img/decoder-network-adapted.png)\n",
" " " "
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 9, "execution_count": 13,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"data": { "data": {
"text/plain": [ "text/plain": [
"tensor(-23351.4941, grad_fn=<SumBackward0>)" "tensor(-23347.4961, grad_fn=<SumBackward0>)"
] ]
}, },
"execution_count": 9, "execution_count": 13,
"metadata": {}, "metadata": {},
"output_type": "execute_result" "output_type": "execute_result"
} }
@@ -371,7 +424,7 @@
" self.relu = nn.LeakyReLU()\n", " self.relu = nn.LeakyReLU()\n",
" self.rnn = nn.GRU(embedding_size, hidden_size)\n", " self.rnn = nn.GRU(embedding_size, hidden_size)\n",
" self.out = nn.Sequential(\n", " self.out = nn.Sequential(\n",
" nn.LeakyReLU(),\n", "# nn.ReLU(),\n",
" nn.Linear(hidden_size, output_size),\n", " nn.Linear(hidden_size, output_size),\n",
" nn.LogSoftmax(2)\n", " nn.LogSoftmax(2)\n",
" )\n", " )\n",
@@ -400,12 +453,12 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"![](img/attention-decoder-network.png)" "![](img/attention-decoder-network-adapted.png)"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 10, "execution_count": 24,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
@@ -421,7 +474,7 @@
"torch.Size([1, 1, 2])" "torch.Size([1, 1, 2])"
] ]
}, },
"execution_count": 10, "execution_count": 24,
"metadata": {}, "metadata": {},
"output_type": "execute_result" "output_type": "execute_result"
} }
@@ -435,7 +488,7 @@
" self.device = device\n", " self.device = device\n",
" self.embedding = nn.Sequential(\n", " self.embedding = nn.Sequential(\n",
" nn.Embedding(output_size, embedding_size),\n", " nn.Embedding(output_size, embedding_size),\n",
" nn.Dropout(dropout)\n", "# nn.Dropout(dropout)\n",
" )\n", " )\n",
" \n", " \n",
" # Seperate neural network to learn the attention weights\n", " # Seperate neural network to learn the attention weights\n",
@@ -487,22 +540,22 @@
"hidden_size = 256\n", "hidden_size = 256\n",
"max_length = 10\n", "max_length = 10\n",
"\n", "\n",
"m = Encoder(eng.n_words, embedding_size, hidden_size, bidirectional=False, device='cpu')\n", "m = Encoder(eng.n_words, embedding_size, hidden_size, bidirectional=False, device=device)\n",
"scentence = torch.tensor([1, 23, 9])\n", "sentence = torch.tensor([1, 23, 9], device=device)\n",
"out, h = m(scentence)\n", "out, h = m(sentence)\n",
"print(out.shape)\n", "print(out.shape)\n",
"\n", "\n",
"encoder_outputs = torch.zeros(max_length, out.shape[-1], device='cpu')\n", "encoder_outputs = torch.zeros(max_length, out.shape[-1], device=device)\n",
"encoder_outputs[:out.shape[0], :out.shape[-1]] = out.view(out.shape[0], -1)\n", "encoder_outputs[:out.shape[0], :out.shape[-1]] = out.view(out.shape[0], -1)\n",
"\n", "\n",
"\n", "\n",
"m = AttentionDecoder(embedding_size, hidden_size, 2, device='cpu')\n", "m = AttentionDecoder(embedding_size, hidden_size, 2, device=device)\n",
"m(torch.tensor([1]), h, encoder_outputs)[0].shape" "m(torch.tensor([1], device=device), h, encoder_outputs)[0].shape"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 11, "execution_count": 16,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
@@ -527,10 +580,11 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 20, "execution_count": 25,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"epochs = 10\n",
"teacher_forcing_ratio = 0.5\n", "teacher_forcing_ratio = 0.5\n",
"\n", "\n",
"embedding_size = 100\n", "embedding_size = 100\n",
@@ -539,12 +593,14 @@
"encoder = Encoder(eng.n_words, embedding_size, context_vector_size, bidirectional)\n", "encoder = Encoder(eng.n_words, embedding_size, context_vector_size, bidirectional)\n",
"context_vector_size = context_vector_size * 2 if bidirectional else context_vector_size \n", "context_vector_size = context_vector_size * 2 if bidirectional else context_vector_size \n",
"decoder = AttentionDecoder(embedding_size, context_vector_size, fra.n_words)\n", "decoder = AttentionDecoder(embedding_size, context_vector_size, fra.n_words)\n",
"writer = SummaryWriter('tb/train3')" "\n",
"if 'SummaryWriter' in globals():\n",
" writer = SummaryWriter('tb/train-3')"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 13, "execution_count": 26,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
@@ -553,7 +609,7 @@
"'cuda'" "'cuda'"
] ]
}, },
"execution_count": 13, "execution_count": 26,
"metadata": {}, "metadata": {},
"output_type": "execute_result" "output_type": "execute_result"
} }
@@ -564,7 +620,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 21, "execution_count": 27,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
@@ -574,7 +630,13 @@
"epoch 0\n", "epoch 0\n",
"epoch 1\n", "epoch 1\n",
"epoch 2\n", "epoch 2\n",
"epoch 3\n" "epoch 3\n",
"epoch 4\n",
"epoch 5\n",
"epoch 6\n",
"epoch 7\n",
"epoch 8\n",
"epoch 9\n"
] ]
} }
], ],
@@ -585,9 +647,6 @@
" optim_encoder = torch.optim.SGD(encoder.parameters(), lr=0.01)\n", " optim_encoder = torch.optim.SGD(encoder.parameters(), lr=0.01)\n",
" optim_decoder = torch.optim.SGD(decoder.parameters(), lr=0.01) \n", " optim_decoder = torch.optim.SGD(decoder.parameters(), lr=0.01) \n",
"\n", "\n",
" epochs = 4\n",
" batch_size = 1\n",
"\n",
" encoder.train(True)\n", " encoder.train(True)\n",
" decoder.train(True)\n", " decoder.train(True)\n",
"\n", "\n",
@@ -614,6 +673,7 @@
" loss = run_decoder(decoder, criterion, fra_sentence, h, teacher_forcing, encoder_outputs)\n", " loss = run_decoder(decoder, criterion, fra_sentence, h, teacher_forcing, encoder_outputs)\n",
"\n", "\n",
" loss.backward()\n", " loss.backward()\n",
" if 'SummaryWriter' in globals():\n",
" writer.add_scalar('loss', loss.cpu().item() / (len(fra_sentence)))\n", " writer.add_scalar('loss', loss.cpu().item() / (len(fra_sentence)))\n",
"\n", "\n",
" optim_decoder.step()\n", " optim_decoder.step()\n",
@@ -626,260 +686,251 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 18, "execution_count": 28,
"metadata": {},
"outputs": [],
"source": [
"m = AttentionDecoder"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {}, "metadata": {},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"English scentence:\t we re all single\n", "English scentence:\t i am playing video games\n",
"French scentence:\t nous sommes toutes celibataires\n", "French scentence:\t je joue a des jeux video\n",
"\n", "\n",
"Model translation:\t nous sommes toutes celibataire \n", "Model translation:\t je joue a a ce \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t he is amusing himself by playing video games\n", "English scentence:\t they re looking for you\n",
"French scentence:\t il s amuse en jouant aux jeux videos\n", "French scentence:\t ils te cherchent\n",
"\n", "\n",
"Model translation:\t il est plein a des anglais \n", "Model translation:\t ils sont en train de \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i m an artist\n", "English scentence:\t i m sure that s wrong\n",
"French scentence:\t je suis une artiste\n", "French scentence:\t je suis sur que que c est mal\n",
"\n", "\n",
"Model translation:\t je suis un artiste \n", "Model translation:\t je suis sur que c est \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t he s enjoying himself\n", "English scentence:\t i m about to go out\n",
"French scentence:\t il s amuse\n", "French scentence:\t je vais sortir\n",
"\n", "\n",
"Model translation:\t il est est a \n", "Model translation:\t je vais sortir de sortir \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t he is an american\n", "English scentence:\t he is on night duty tonight\n",
"French scentence:\t il est americain\n", "French scentence:\t il travaille de nuit ce soir\n",
"\n", "\n",
"Model translation:\t il est un \n", "Model translation:\t il est au soir sur ce \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t you re the oldest\n", "English scentence:\t i m not saying anything\n",
"French scentence:\t tu es le plus vieux\n", "French scentence:\t je ne dis rien\n",
"\n", "\n",
"Model translation:\t vous etes le plus \n", "Model translation:\t je ne suis pas du \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t he s about to leave\n", "English scentence:\t we re not lost\n",
"French scentence:\t il est sur le point de partir\n", "French scentence:\t nous ne sommes pas perdues\n",
"\n", "\n",
"Model translation:\t il est sur le point \n", "Model translation:\t nous ne sommes pas \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i m really happy\n", "English scentence:\t you re very funny\n",
"French scentence:\t je suis vraiment heureuse\n", "French scentence:\t tu es tres drole\n",
"\n", "\n",
"Model translation:\t je suis vraiment contente \n", "Model translation:\t tu es tres droles \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t we re all hungry\n", "English scentence:\t i m writing a novel\n",
"French scentence:\t nous avons tous faim\n", "French scentence:\t j ecris un roman\n",
"\n", "\n",
"Model translation:\t nous avons toutes faim \n", "Model translation:\t j ecris en roman \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i m not giving you any more money\n", "English scentence:\t we re all busy\n",
"French scentence:\t je ne te donnerai pas davantage d argent\n", "French scentence:\t nous sommes toutes occupees\n",
"\n", "\n",
"Model translation:\t je ne te donnerai d argent d argent \n", "Model translation:\t nous sommes tous occupes \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t he is independent of his parents\n", "English scentence:\t you re fired\n",
"French scentence:\t il est independant de ses parents\n", "French scentence:\t tu es vire\n",
"\n", "\n",
"Model translation:\t il est plein de ses parents \n", "Model translation:\t tu es licencie \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i am not a doctor but a teacher\n", "English scentence:\t he is likely to win the game\n",
"French scentence:\t je ne suis pas medecin mais professeur\n", "French scentence:\t il est probable qu il remporte la partie\n",
"\n", "\n",
"Model translation:\t je ne suis pas medecin mais enseignant \n", "Model translation:\t il a des chances de remporter le \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t you are responsible for the result\n", "English scentence:\t i m afraid for his life\n",
"French scentence:\t vous etes responsable des resultats\n", "French scentence:\t je crains pour sa vie\n",
"\n", "\n",
"Model translation:\t vous es responsable de l \n", "Model translation:\t je crains pour sa vie sa \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i m sympathetic\n", "English scentence:\t we re going to work tonight\n",
"French scentence:\t j eprouve de la compassion\n", "French scentence:\t nous allons travailler ce soir\n",
"\n", "\n",
"Model translation:\t je ne suis \n", "Model translation:\t nous allons travailler ce soir \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t you are drunk\n", "English scentence:\t she is not quite content\n",
"French scentence:\t tu es saoul\n", "French scentence:\t elle n est pas tout a fait satisfaite\n",
"\n", "\n",
"Model translation:\t vous etes impressionnee \n", "Model translation:\t elle n est pas tout \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t you aren t as short as i am\n", "English scentence:\t i m a tv addict\n",
"French scentence:\t tu n es pas aussi petite que moi\n", "French scentence:\t je suis accro a la tele\n",
"\n", "\n",
"Model translation:\t vous n etes pas aussi petit que moi \n", "Model translation:\t je suis un a la \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i m resilient\n", "English scentence:\t we re not so sure\n",
"French scentence:\t je suis endurante\n", "French scentence:\t nous n en sommes pas si sures\n",
"\n", "\n",
"Model translation:\t je suis endurant \n", "Model translation:\t nous n en sommes pas \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t you re wrong again\n", "English scentence:\t she stopped talking\n",
"French scentence:\t vous avez a nouveau tort\n", "French scentence:\t elle arreta de parler\n",
"\n", "\n",
"Model translation:\t tu as tort \n", "Model translation:\t elle a arrete \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i am thinking of my vacation\n", "English scentence:\t i m out of ammo\n",
"French scentence:\t je pense a mes vacances\n", "French scentence:\t je suis a court de munitions\n",
"\n", "\n",
"Model translation:\t je songe de mes enfants \n", "Model translation:\t je suis a court de \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t she is able to sing very well\n", "English scentence:\t i m sorry i don t recognize you\n",
"French scentence:\t elle sait tres bien chanter\n", "French scentence:\t je suis desolee je ne te reconnais pas\n",
"\n", "\n",
"Model translation:\t elle va bien bien bien bien \n", "Model translation:\t je suis desole je vous remets pas \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i m lazy\n", "English scentence:\t you re very brave\n",
"French scentence:\t je suis faineant\n", "French scentence:\t tu es tres brave\n",
"\n", "\n",
"Model translation:\t je suis paresseux \n", "Model translation:\t vous etes fort brave \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t he is a cruel person\n", "English scentence:\t she is older and wiser now\n",
"French scentence:\t c est un homme cruel\n", "French scentence:\t elle est plus agee et plus sage maintenant\n",
"\n", "\n",
"Model translation:\t il est une personne \n", "Model translation:\t elle est plus et et plus \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t he is a poet\n", "English scentence:\t you re rude\n",
"French scentence:\t c est un poete\n", "French scentence:\t vous etes grossieres\n",
"\n", "\n",
"Model translation:\t il est un \n", "Model translation:\t vous etes grossiers \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t we re plastered\n", "English scentence:\t she s a jealous woman\n",
"French scentence:\t nous sommes bourres\n", "French scentence:\t c est une femme jalouse\n",
"\n", "\n",
"Model translation:\t nous sommes bourrees \n", "Model translation:\t c est une femme jalouse \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t she suddenly fell silent\n", "English scentence:\t i m going to reconsider it\n",
"French scentence:\t elle se tut soudain\n", "French scentence:\t je vais y repenser encore une fois\n",
"\n", "\n",
"Model translation:\t elle est deux deux \n", "Model translation:\t je vais y aller \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t you re not making this easy\n", "English scentence:\t i m not happy about it\n",
"French scentence:\t vous ne rendez pas ca facile\n", "French scentence:\t je n en suis pas contente\n",
"\n", "\n",
"Model translation:\t vous ne fais pas ca \n", "Model translation:\t je n en suis pas heureux \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t she s very afraid of dogs\n", "English scentence:\t you re not the first\n",
"French scentence:\t elle a une peur bleue des chiens\n", "French scentence:\t vous n etes pas le premier\n",
"\n", "\n",
"Model translation:\t elle a tres peur des chiens \n", "Model translation:\t tu n es pas le \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t she is living abroad\n", "English scentence:\t i m just following orders\n",
"French scentence:\t elle vit actuellement a l etranger\n", "French scentence:\t je ne fais qu obeir aux ordres\n",
"\n", "\n",
"Model translation:\t elle est l a \n", "Model translation:\t je suis fais d \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i m going to my grandmother s\n", "English scentence:\t i m on crutches for the next month\n",
"French scentence:\t je vais chez ma grand mere\n", "French scentence:\t je suis en bequilles pour un mois\n",
"\n", "\n",
"Model translation:\t je vais me mon de la \n", "Model translation:\t je suis en bequilles pour le mois \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t they re not always right\n", "English scentence:\t i m in charge of security\n",
"French scentence:\t elles n ont pas toujours raison\n", "French scentence:\t je suis responsable de la securite\n",
"\n", "\n",
"Model translation:\t elles n ont pas toujours \n", "Model translation:\t je suis la la la la \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t you are very attractive in blue\n", "English scentence:\t she s young enough to be your daughter\n",
"French scentence:\t le bleu vous va tres bien\n", "French scentence:\t elle est assez jeune pour etre ta fille\n",
"\n", "\n",
"Model translation:\t vous avez vraiment beaucoup a \n", "Model translation:\t elle est assez jeune pour etre ta fille \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t you re winning aren t you\n", "English scentence:\t you re a jolly good feller\n",
"French scentence:\t vous gagnez n est ce pas\n", "French scentence:\t t es un joyeux drille\n",
"\n", "\n",
"Model translation:\t vous etes est n est ce \n", "Model translation:\t t es un sacre \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i m ticklish\n", "English scentence:\t he s a bit rough around the edges\n",
"French scentence:\t je suis chatouilleuse\n", "French scentence:\t il est un peu rugueux\n",
"\n", "\n",
"Model translation:\t je suis mal \n", "Model translation:\t il est un peu peu \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i m not going to name names\n", "English scentence:\t we re giving up\n",
"French scentence:\t je ne vais pas citer de noms\n", "French scentence:\t nous abandonnons\n",
"\n", "\n",
"Model translation:\t je ne vais pas m a \n", "Model translation:\t nous abandonnons \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t they are at broadway avenue\n", "English scentence:\t they are very big\n",
"French scentence:\t ils sont au broadway avenue\n", "French scentence:\t ils sont tres grands\n",
"\n", "\n",
"Model translation:\t ils sont a l des \n", "Model translation:\t ils sont tres gros \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i m sure\n", "English scentence:\t i m puzzled\n",
"French scentence:\t j en suis sure\n", "French scentence:\t je suis perplexe\n",
"\n", "\n",
"Model translation:\t je suis certain \n", "Model translation:\t je suis en \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t she said that she was happy\n", "English scentence:\t he is no longer a child\n",
"French scentence:\t elle a dit qu elle etait heureuse\n", "French scentence:\t ce n est plus un enfant\n",
"\n", "\n",
"Model translation:\t elle dit qu a heureuse \n", "Model translation:\t ce n est plus un enfant \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i m all done\n", "English scentence:\t we re up early\n",
"French scentence:\t j ai tout fini\n", "French scentence:\t nous sommes debout tot\n",
"\n", "\n",
"Model translation:\t je ai tout termine \n", "Model translation:\t nous sommes tot tot \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t we re the problem\n", "English scentence:\t i am going to be fourteen\n",
"French scentence:\t nous sommes le probleme\n", "French scentence:\t je vais avoir quatorze ans\n",
"\n", "\n",
"Model translation:\t nous sommes le probleme \n", "Model translation:\t je vais a tout \n",
"\n", "\n",
"\n", "\n",
"English scentence:\t i m being serious\n", "English scentence:\t i m sorry i missed your birthday\n",
"French scentence:\t je suis serieux\n", "French scentence:\t je suis desolee d avoir rate ton anniversaire\n",
"\n", "\n",
"Model translation:\t je suis serieux \n", "Model translation:\t je suis desole d avoir rate votre \n",
"\n", "\n",
"\n" "\n"
] ]
@@ -917,15 +968,6 @@
" \n", " \n",
"translate(20, 60)" "translate(20, 60)"
] ]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"decoder"
]
} }
], ],
"metadata": { "metadata": {

BIN
seq2seq/img/attention-decoder-network-adapted.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

BIN
seq2seq/img/decoder-network-adapted.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 20 KiB

BIN
seq2seq/img/hello-lead.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.5 KiB

BIN
seq2seq/img/seq2seq.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 22 KiB