daviddoji/Fastbook_Julia
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

clean

Browse Source
This commit is contained in:
Jeremy Howard 2020-09-03 15:51:00 -07:00
parent 40a755b822
commit d67a599255
22 changed files with 669 additions and 201 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
app_blog.ipynb
View File

@ -1,17 +1,5 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "raw",
"metadata": {},
@ -21,17 +9,6 @@
"[role=\"Creating a blog\"]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastbook import *\n",
"from fastai.vision.widgets import *"
]
},
{
"cell_type": "markdown",
"metadata": {},
@ -76,7 +53,7 @@
"metadata": {},
"source": [
"You’ll need an account on GitHub, so head over there now and create an account if you don’t have one already. Make sure that you are logged in. Normally, GitHub is used by software developers for writing code, and they use a sophisticated command-line tool to work with it—but we're going to show you an approach that doesn't use the command line at all!\n",
"\n",_re
"\n",
"To get started, point your browser to [https://github.com/fastai/fast_template/generate](https://github.com/fastai/fast_template/generate) (you need to be logged in to GitHub for the link to work). This will allow you to create a place to store your blog, called a *repository*. You will a screen like the one in <<github_repo>>. Note that you have to enter your repository name using the *exact* format shown here—that is, your GitHub username followed by `.github.io`.\n",
"\n",
"<img width=\"440\" src=\"images/fast_template/image1.png\" id=\"github_repo\" caption=\"Creating your repository\" alt=\"Screenshot of the GitHub page for creating a new repository\">\n",
@ -269,6 +246,31 @@
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": false,
"sideBar": true,
"skip_h1_title": true,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,

39
clean/01_intro.ipynb
View File

@ -7,7 +7,19 @@
"outputs": [],
"source": [
"#hide\n",
"from utils import *"
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastbook import *"
]
},
{
@ -1569,6 +1581,31 @@
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": false,
"sideBar": true,
"skip_h1_title": true,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,

42
clean/02_production.ipynb
View File

@ -7,7 +7,19 @@
"outputs": [],
"source": [
"#hide\n",
"from utils import *\n",
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastbook import *\n",
"from fastai.vision.widgets import *"
]
},
@ -99,6 +111,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# clean\n",
"To download images with Bing Image Search, sign up at Microsoft for a free account. You will be given a key, which you can copy and enter in a cell as follows (replacing 'XXX' with your key and executing it):"
]
},
@ -108,7 +121,7 @@
"metadata": {},
"outputs": [],
"source": [
"key = 'XXX'"
"key = os.environ.get('AZURE_SEARCH_KEY', 'XXX')"
]
},
{
@ -1087,6 +1100,31 @@
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": false,
"sideBar": true,
"skip_h1_title": true,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,

37
clean/03_ethics.ipynb
View File

@ -1,5 +1,17 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "markdown",
"metadata": {},
@ -304,6 +316,31 @@
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": false,
"sideBar": true,
"skip_h1_title": true,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,

39
clean/04_mnist_basics.ipynb
View File

@ -1,5 +1,17 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
@ -8,7 +20,7 @@
"source": [
"#hide\n",
"from fastai.vision.all import *\n",
"from utils import *\n",
"from fastbook import *\n",
"\n",
"matplotlib.rc('image', cmap='Greys')"
]
@ -4294,6 +4306,31 @@
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": false,
"sideBar": true,
"skip_h1_title": true,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,

39
clean/05_pet_breeds.ipynb
View File

@ -7,7 +7,19 @@
"outputs": [],
"source": [
"#hide\n",
"from utils import *"
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastbook import *"
]
},
{
@ -1762,6 +1774,31 @@
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": false,
"sideBar": true,
"skip_h1_title": true,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,

353
clean/06_multicat.ipynb
View File

File diff suppressed because one or more lines are too long

14
clean/07_sizing_and_tta.ipynb
View File

@ -7,7 +7,19 @@
"outputs": [],
"source": [
"#hide\n",
"from utils import *"
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastbook import *"
]
},
{

39
clean/08_collab.ipynb
View File

@ -7,7 +7,19 @@
"outputs": [],
"source": [
"#hide\n",
"from utils import *"
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastbook import *"
]
},
{
@ -1739,6 +1751,31 @@
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": false,
"sideBar": true,
"skip_h1_title": true,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,

39
clean/09_tabular.ipynb
View File

@ -1,5 +1,17 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
@ -17,7 +29,7 @@
],
"source": [
"#hide\n",
"from utils import *\n",
"from fastbook import *\n",
"from kaggle import api\n",
"from pandas.api.types import is_string_dtype, is_numeric_dtype, is_categorical_dtype\n",
"from fastai.tabular.all import *\n",
@ -8443,6 +8455,31 @@
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": false,
"sideBar": true,
"skip_h1_title": true,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,

18
clean/11_midlevel_data.ipynb
View File

@ -7,7 +7,19 @@
"outputs": [],
"source": [
"#hide\n",
"from utils import *\n",
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastbook import *\n",
"from IPython.display import display,HTML"
]
},
@ -619,7 +631,7 @@
"metadata": {},
"outputs": [],
"source": [
"class SiameseImage(Tuple):\n",
"class SiameseImage(fastuple):\n",
" def show(self, ctx=None, **kwargs): \n",
" img1,img2,same_breed = self\n",
" if not isinstance(img1, Tensor):\n",
@ -860,7 +872,7 @@
"source": [
"Congratulations—you've completed all of the chapters in this book that cover the key practical parts of training models and using deep learning! You know how to use all of fastai's built-in applications, and how to customize them using the data block API and loss functions. You even know how to create a neural network from scratch, and train it! (And hopefully you now know some of the questions to ask to make sure your creations help improve society too.)\n",
"\n",
"The knowledge you already have is enough to create full working prototypes of many types of neural network application. More importantly, it will help you understand the capabilities and limitations of deep learning models, and how to design a system that's well adapted to them.\n",
"The knowledge you already have is enough to create full working prototypes of many types of neural network applications. More importantly, it will help you understand the capabilities and limitations of deep learning models, and how to design a system that's well adapted to them.\n",
"\n",
"In the rest of this book we will be pulling apart those applications, piece by piece, to understand the foundations they are built on. This is important knowledge for a deep learning practitioner, because it is what allows you to inspect and debug models that you build and create new applications that are customized for your particular projects."
]

14
clean/12_nlp_dive.ipynb
View File

@ -7,7 +7,19 @@
"outputs": [],
"source": [
"#hide\n",
"from utils import *"
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastbook import *"
]
},
{

14
clean/13_convolutions.ipynb
View File

@ -1,5 +1,17 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
@ -8,7 +20,7 @@
"source": [
"#hide\n",
"from fastai.vision.all import *\n",
"from utils import *\n",
"from fastbook import *\n",
"\n",
"matplotlib.rc('image', cmap='Greys')"
]

14
clean/14_resnet.ipynb
View File

@ -1,5 +1,17 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
@ -9,7 +21,7 @@
"outputs": [],
"source": [
"#hide\n",
"from utils import *"
"from fastbook import *"
]
},
{

18
clean/15_arch_details.ipynb
View File

@ -7,7 +7,19 @@
"outputs": [],
"source": [
"#hide\n",
"from utils import *"
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastbook import *"
]
},
{
@ -111,7 +123,7 @@
"path = untar_data(URLs.PETS)\n",
"files = get_image_files(path/\"images\")\n",
"\n",
"class SiameseImage(Tuple):\n",
"class SiameseImage(fastuple):\n",
" def show(self, ctx=None, **kwargs): \n",
" img1,img2,same_breed = self\n",
" if not isinstance(img1, Tensor):\n",
@ -387,7 +399,7 @@
"1. What is `model_meta`? Try printing it to see what's inside.\n",
"1. Read the source code for `create_head` and make sure you understand what each line does.\n",
"1. Look at the output of `create_head` and make sure you understand why each layer is there, and how the `create_head` source created it.\n",
"1. Figure out how to change the dropout, layer size, and number of layers created by `create_cnn`, and see if you can find values that result in better accuracy from the pet recognizer.\n",
"1. Figure out how to change the dropout, layer size, and number of layers created by `cnn_learner`, and see if you can find values that result in better accuracy from the pet recognizer.\n",
"1. What does `AdaptiveConcatPool2d` do?\n",
"1. What is \"nearest neighbor interpolation\"? How can it be used to upsample convolutional activations?\n",
"1. What is a \"transposed convolution\"? What is another name for it?\n",

12
clean/17_foundations.ipynb
View File

@ -1,5 +1,17 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,

14
clean/18_CAM.ipynb
View File

@ -1,5 +1,17 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
@ -9,7 +21,7 @@
"outputs": [],
"source": [
"#hide\n",
"from utils import *"
"from fastbook import *"
]
},
{

14
clean/19_learner.ipynb
View File

@ -7,7 +7,19 @@
"outputs": [],
"source": [
"#hide\n",
"from utils import *"
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"from fastbook import *"
]
},
{

12
clean/20_conclusion.ipynb
View File

@ -1,5 +1,17 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "markdown",
"metadata": {},

12
clean/app_jupyter.ipynb
View File

@ -1,5 +1,17 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#hide\n",
"!pip install -Uqq fastbook\n",
"import fastbook\n",
"fastbook.setup_book()"
]
},
{
"cell_type": "markdown",
"metadata": {},

BIN
tools/.clean.py.swp Normal file
View File

Binary file not shown.

37
tools/clean.py Normal file
View File

@ -0,0 +1,37 @@
import nbformat
from nbdev.export import read_nb
from fastcore.all import *
_re_header = re.compile(r'^#+\s+\S+')
_re_clean = re.compile(r'^\s*#\s*clean\s*')
def is_header_cell(cell): return _re_header.search(cell['source']) is not None
def is_clean_cell(cell): return _re_clean.search(cell['source']) is not None
_re_questionnaire = re.compile(r'^#+\s+Questionnaire')
def get_stop_idx(cells):
i = 0
while i < len(cells) and _re_questionnaire.search(cells[i]['source']) is None: i+=1
return i
def clean_tags(cell):
if is_header_cell(cell): return cell
for attr in ["id", "caption", "alt", "width", "hide_input", "hide_output", "clean"]:
cell["source"] = re.sub(r'#\s*' + attr + r'.*?($|\n)', '', cell["source"])
return cell
def clean_nb(fname, dest):
nb = read_nb(fname)
i = get_stop_idx(nb['cells'])
nb['cells'] = [clean_tags(c) for j,c in enumerate(nb['cells']) if
c['cell_type']=='code' or is_header_cell(c) or is_clean_cell(c) or j >= i]
with open(dest/fname.name, 'w') as f: nbformat.write(nb, f, version=4)
def clean_all(path='.', dest_path='clean'):
path,dest_path = Path(path),Path(dest_path)
fns = [f for f in path.iterdir() if f.suffix == '.ipynb' and not f.name.startswith('_')]
for fn in fns: clean_nb(fn, dest=dest_path)
if __name__=='__main__': clean_all()