|
1 | 1 |
|
2 |
| -<div align="center"> |
3 | 2 |
|
4 |
| -# 🐍 Python Machine Learning Template |
| 3 | +# Python-Machine-Learning-Template |
5 | 4 |
|
6 |
| -[](https://choosealicense.com/licenses/mit/) |
7 |
| -[](https://github.com/psf/black) |
8 |
| -[](https://github.com/pre-commit/pre-commit) |
9 | 5 |
|
10 |
| -[]() |
11 |
| -[]() |
12 |
| -[]() |
13 |
| - |
14 |
| - |
15 |
| -The Python Machine Learning Template is designed to provide a comprehensive structure for end-to-end |
16 |
| -**Machine Learning** projects in Python. Whether you're working on ***Computer Vision***, ***Natural Language Processing***, ***Reinforcement Learning***, or ***traditional Machine Learning/Data Science***, this template offers a simple and intuitive way to organize and manage your project. |
17 |
| - |
18 |
| - |
19 |
| -<a href="https://github.com/anujonthemove/Python-Machine-Learning-Template/generate"><img src="https://raw.githubusercontent.com/anujonthemove/Python-Machine-Learning-Template/main/.assets/images/do-not-clone_fork-click-this-button-to-use-the-template.svg" alt="click this button to use the template"></a> |
20 |
| - |
21 |
| -To get started, simply click on the above button 👆 or <b><font color="red">"Use this template"</font></b> button at the top and create your project based on this template. |
22 |
| - |
23 |
| -<a href="https://anujonthemove.github.io/Software-Engineering-Handbook/python-machine-learning-template/"><img src="https://raw.githubusercontent.com/anujonthemove/Python-Machine-Learning-Template/main/.assets/images/check-detailed-documentation-here.svg" alt="check detailed documentation here"></a> |
24 |
| - |
25 |
| -</div> |
26 |
| - |
27 |
| - |
28 |
| -The following sections provide an overview of the directory structure and instructions for setting up your project workspace. |
29 |
| - |
30 |
| -## 🗂️ Directory Structure |
31 |
| - |
32 |
| -``` |
33 |
| -
|
34 |
| -. |
35 |
| -├── config/ <- 📂 Configuration files [.ini, .json, .yaml] |
36 |
| -├── data/ <- 📂 Images, numpy data objects, text files |
37 |
| -├── docs/ <- 📂 Store .md files. Used by Mkdocs for Project Documentation |
38 |
| -├── helpers/ <- 📂 Utility/helper files/modules for the project |
39 |
| -├── html/ <- 📂 Store .html files and accompanying assets. Used by pdoc3 for API Documentation |
40 |
| -├── logs/ <- 📂 Log files generated by the project during execution |
41 |
| -├── models/ <- 📂 Model files [.h5, .pkl, .pt] - pre-trained weight files, snapshots, checkpoints |
42 |
| -├── notebooks/ <- 📂 Jupyter Notebooks |
43 |
| -├── references/ <- 📂 Data dictionaries, manuals, and all other explanatory materials |
44 |
| -├── scripts/ <- 📂 Utility scripts for various project-related tasks |
45 |
| -├── src/ <- 📂 Source code (.py files) |
46 |
| -├── tests/ <- 📂 Unit tests for the project |
47 |
| -├── workspaces/ <- 📂 Multi-user workspace that can be used in the case of a single machine |
48 |
| -├── .env-template <- 🔧 Template for the .env file |
49 |
| -├── .gitattributes <- 🔧 Standard .gitattributes file |
50 |
| -├── .gitignore <- 📛 Standard .gitignore file |
51 |
| -├── .pre-commit-config.yaml <- 🔧 Config file for Git Hooks |
52 |
| -├── LICENSE <- 🪧 License file [choose your appropriate license from GitHub] |
53 |
| -├── mkdocs.yml <- 🗞️ Base config file required for Mkdocs |
54 |
| -├── Pipfile <- 🗃️ Most commonly used python packages |
55 |
| -├── project_setup.bat <- 📜 Project script for Windows OS |
56 |
| -├── project_setup.sh <- 📜 Project script for Linux/MacOS |
57 |
| -├── README.md <- 📝 Project readme |
58 |
| -├── setup.py <- 📦️ For installing & packaging the project |
59 |
| -└── tox.ini <- 🔧 General-purpose package configuration manager |
60 |
| -
|
61 |
| -``` |
62 |
| - |
63 |
| -## 🚀 Features |
64 |
| - |
65 |
| -* 🤓 Simple, intuitive, yet comprehensive directory structure for organizing your machine learning project. |
66 |
| - |
67 |
| -* 😎 Setup script provided for Windows, Mac, and Linux. Check [Setup](#%EF%B8%8F-setup) section for more details. |
68 |
| - |
69 |
| -* 🤗 Only requires native Python and Virtual Environment package installed. |
70 |
| - |
71 |
| -* 🤩 Uses [Pipenv](https://pipenv.pypa.io/en/latest/#). Here are some of the advantages of using it: |
72 |
| - |
73 |
| - * **💪 Consolidated Tooling:** By combining the functionalities of pip, virtualenv, and package management, Pipenv eliminates the need to use these tools separately. |
74 |
| - |
75 |
| - * **🤌 Simplified Dependency Management:** Pipenv removes the need for the requirements.txt file and replaces it with the Pipfile, which effectively tracks dependencies. Additionally, Pipenv utilizes the Pipfile.lock to ensure consistent and reliable application builds. |
76 |
| - |
77 |
| - * **🤟 Cross-Platform Compatibility:** Pipenv supports a wide range of operating systems, including Linux, macOS, and Windows. |
78 |
| - |
79 |
| - |
80 |
| -## 🛠️ Setup |
81 |
| - |
82 |
| -Project setup scripts have been provided separately for Linux/MacOS and Windows OS. |
83 |
| - |
84 |
| -Checkout the demo quick setup. |
85 |
| - |
86 |
| -### 🎥 Demo |
87 |
| - |
88 |
| -[](https://asciinema.org/a/aLntFu3A4w0JWivMfXLzfVvCv) |
89 |
| - |
90 |
| - |
91 |
| -<details> |
92 |
| -<summary> <h3> Command details </h3> </summary> |
93 |
| - |
94 |
| - ``` |
95 |
| - ## For Linux/MacOS |
96 |
| - source project_setup.sh [OPTIONS] |
97 |
| -
|
98 |
| - or |
99 |
| -
|
100 |
| - ## For Windows OS |
101 |
| - project_setup.bat [OPTIONS] |
102 |
| - ``` |
103 |
| - |
104 |
| - Replace [OPTIONS] with any combination of the following options: |
105 |
| - |
106 |
| - * `--install`: **Required argument**. If nothing is passed, a help message is displayed. |
107 |
| - |
108 |
| - * `--install-dev`: Optional argument. Pass this flag along with `--install` flag to install development packages. |
109 |
| - |
110 |
| - * `--use-proxy`: Optional argument. This flag enables installation of python packages behind proxy. Check Using .env section for proxy configuration. |
111 |
| - |
112 |
| - * `--unset-proxy`: Optional argument. This flag disables proxy. |
113 |
| - |
114 |
| - * `--clear-readme`: Optional argument. Clear README.md file after setting up the project. |
115 |
| - * 📣 ***Caution: Use this only when you are setting up the project for the first time.*** |
116 |
| - |
117 |
| - * `--remove-cache`: Optional argument. Removes `pip` and `pipenv` cache files. |
118 |
| - * 💡 ***Use this to clear cache files generated during package installation*** |
119 |
| - |
120 |
| - * `--help`: Display the help message. |
121 |
| - |
122 |
| -</details> |
123 |
| - |
124 |
| - |
125 |
| -<details> |
126 |
| -<summary><h3> 🐧 Instructions for Linux/MacOS </h3></summary> |
127 |
| - |
128 |
| -For setting up the project, `project_setup.sh` script has been provided along with some options. |
129 |
| - |
130 |
| - |
131 |
| -### 🧑💻 Steps: |
132 |
| - |
133 |
| -1. Open terminal and navigate to your project directory. |
134 |
| - |
135 |
| - **Case (a): Setting up in _Development_ environment** |
136 |
| - |
137 |
| - If you are setting up the project inside **development** environment, use: |
138 |
| - |
139 |
| - ``` |
140 |
| - source project_setup.sh --install --install-dev |
141 |
| - ``` |
142 |
| -
|
143 |
| - Incase you are working behind a proxy, use the following command instead: |
144 |
| - |
145 |
| - ``` |
146 |
| - source project_setup.sh --install --install-dev --use-proxy |
147 |
| - ``` |
148 |
| - |
149 |
| -
|
150 |
| - **Case (b): If you are setting up the project in _production_ environment**, |
151 |
| - |
152 |
| - If you are setting up the project inside **production** environment, you may only require base packages to be installaed, use: |
153 |
| -
|
154 |
| - ``` |
155 |
| - source project_setup.sh --install |
156 |
| - ``` |
157 |
| -
|
158 |
| - If you are working behind a proxy, use the following command: |
159 |
| - |
160 |
| - ``` |
161 |
| - source project_setup.sh --install --use-proxy |
162 |
| - ``` |
163 |
| -
|
164 |
| -2. If you are setting up the project first time using this template, then you should replace contents of the README.md with the name of your project: |
165 |
| -
|
166 |
| - ``` |
167 |
| - source project_setup.sh --clear-readme |
168 |
| - ``` |
169 |
| - |
170 |
| - ***Use this command only once in the development environment. DO NOT run this once you write your own readme. Also, do not run this in production environment.*** |
171 |
| -
|
172 |
| -
|
173 |
| -
|
174 |
| -#### 📝 Important Note |
175 |
| -
|
176 |
| -* For any other package installation apart from the listed packages in `Pipfile` use `pipenv` as follows: |
177 |
| -
|
178 |
| - ``` |
179 |
| - pipenv install package_name |
180 |
| - ``` |
181 |
| -
|
182 |
| - By default, `pipenv` loads all the `.env` variables, therefore you need to unset the proxy first if you are not behind proxy. |
183 |
| -
|
184 |
| - Use the following command: |
185 |
| -
|
186 |
| - ``` |
187 |
| - source project_setup.sh --unset-proxy |
188 |
| - ``` |
189 |
| - You should then be able to install packages using pipenv as stated above. |
190 |
| -
|
191 |
| -* During package installation, the packages are downloaded and cached. This consumes a lot of disk, hence you should clear pip and pipenv cache from time to time. Use the following command: |
192 |
| -
|
193 |
| - ``` |
194 |
| - source project_setup.sh --remove-cache |
195 |
| - ``` |
196 |
| -
|
197 |
| -
|
198 |
| -* ✅ To ensure a conflict-free environment setup, it is strongly recommended to always run the `project_setup.sh` script to create a virtual environment for your project. |
199 |
| -
|
200 |
| -* ❗You should run the script **ONLY** using the `source` command to ensure that the virtual environment `.venv` is automatically activated at the end of setup in the current shell session. |
201 |
| -
|
202 |
| -
|
203 |
| -</details> |
204 |
| -<details> |
205 |
| -
|
206 |
| -<summary><h3> 🪟 Instructions for Windows OS</h3></summary> |
207 |
| -
|
208 |
| -
|
209 |
| -For setting up the project, `project_setup.bat` script has been provided along with some options. |
210 |
| -
|
211 |
| -
|
212 |
| -### 🧑💻 Steps: |
213 |
| -
|
214 |
| -1. Open Command Prompt (CMD) and navigate to your project directory. |
215 |
| -
|
216 |
| - **Case (a): Setting up in _Development_ environment** |
217 |
| - |
218 |
| - If you are setting up the project inside **development** environment, use: |
219 |
| -
|
220 |
| - ``` |
221 |
| - project_setup.bat --install --install-dev |
222 |
| - ``` |
223 |
| -
|
224 |
| - Incase you are working behind a proxy, use the following command instead: |
225 |
| - |
226 |
| - ``` |
227 |
| - project_setup.bat --install --install-dev --use-proxy |
228 |
| - ``` |
229 |
| - |
230 |
| -
|
231 |
| - **Case (b): If you are setting up the project in _production_ environment**, |
232 |
| - |
233 |
| - If you are setting up the project inside **production** environment, you may only require base packages to be installaed, use: |
234 |
| -
|
235 |
| - ``` |
236 |
| - project_setup.bat --install |
237 |
| - ``` |
238 |
| -
|
239 |
| - If you are working behind a proxy, use the following command: |
240 |
| - |
241 |
| - ``` |
242 |
| - project_setup.bat --install --use-proxy |
243 |
| - ``` |
244 |
| -
|
245 |
| -2. If you are setting up the project first time using this template, then you should replace contents of the README.md with the name of your project: |
246 |
| -
|
247 |
| - ``` |
248 |
| - project_setup.bat --clear-readme |
249 |
| - ``` |
250 |
| - |
251 |
| - ***Use this command only once in the development environment. DO NOT run this once you write your own readme. Also, do not run this in production environment.*** |
252 |
| -
|
253 |
| -
|
254 |
| -
|
255 |
| -#### 📝 Important Note |
256 |
| -
|
257 |
| -* For any other package installation apart from the listed packages in `Pipfile` use `pipenv` as follows: |
258 |
| -
|
259 |
| - ``` |
260 |
| - pipenv install package_name |
261 |
| - ``` |
262 |
| -
|
263 |
| - By default, `pipenv` loads all the `.env` variables, therefore you need to unset the proxy first if you are not behind proxy. |
264 |
| -
|
265 |
| - Use the following command: |
266 |
| -
|
267 |
| - ``` |
268 |
| - project_setup.bat --unset-proxy |
269 |
| - ``` |
270 |
| - You should then be able to install packages using pipenv as stated above. |
271 |
| -
|
272 |
| -* During package installation, the packages are downloaded and cached. This consumes a lot of disk, hence you should clear pip and pipenv cache from time to time. Use the following command: |
273 |
| -
|
274 |
| - ``` |
275 |
| - project_setup.bat --remove-cache |
276 |
| - ``` |
277 |
| -
|
278 |
| -* ✅ To ensure a conflict-free environment setup, it is strongly recommended to always run the `project_setup.bat` script to create a virtual environment for your project. |
279 |
| -
|
280 |
| -* ❗For security reasons, organizations may prevent running .bat scripts on PowerShell. You should run the script **ONLY** on Command Prompt (CMD) to ensure that everything runs without any errors. |
281 |
| -
|
282 |
| -
|
283 |
| -
|
284 |
| -</details> |
285 |
| -
|
286 |
| -
|
287 |
| -## 📦 Packages |
288 |
| -All the packages to be installed are included in the Pipfile. For installing additional packages **ONLY** `pipenv` should be used. |
289 |
| -
|
290 |
| -<details> |
291 |
| -<summary> <h3> Base Packages </h3> </summary> |
292 |
| -
|
293 |
| -``` |
294 |
| -* numpy <- for numerical computing and scientific computing |
295 |
| -* scipy <- mathematical algorithms and convenience functions built on the NumPy |
296 |
| -* pandas <- for data manipulation and analysis |
297 |
| -* matplotlib <- plotting library |
298 |
| -* seaborn <- data visualization library for drawing informative statistical graphics. |
299 |
| -* scikit-learn <- machine learning library |
300 |
| -* jupyter <- web-based interactive computing platform |
301 |
| -* jupyter-server <- backend for Jupyter notebooks. Required when running notebooks in VS Code |
302 |
| -* ipykernel <- interactive Python shell. Required when running notebooks in VS Code |
303 |
| -* ipython <- provides a powerful interactive shell and a kernel for Jupyter |
304 |
| -``` |
305 |
| -</details> |
306 |
| - |
307 |
| -<details> |
308 |
| -<summary> <h3> Development Packages </h3> </summary> |
309 |
| -
|
310 |
| -
|
311 |
| -``` |
312 |
| -* isort <- sorts imports in a python file |
313 |
| -* python-decouple <- Reads configuration/settings from .env, system environment variables |
314 |
| -* flake8 <- Code linter (format checker) |
315 |
| -* flake8-tabs <- Tab (and Spaces) Style Checker for flake8 |
316 |
| -* black <- Code formatter |
317 |
| -* mypy <- Static type checker |
318 |
| -* pre-commit <- A framework for managing and maintaining multi-language pre-commit hooks. |
319 |
| -* pdoc3 <- Generate API documentation for Python projects |
320 |
| -* mkdocs <- Generate Project documentation for Python projects |
321 |
| -``` |
322 |
| -</details> |
323 |
| -
|
324 |
| -
|
325 |
| -## 🌟 Star Us |
326 |
| -If you find our project useful, please consider giving it a star on GitHub. 🤩 |
327 |
| -
|
328 |
| -It motivates us to continue improving and adding new features. 💪 |
329 |
| -
|
330 |
| -Thank you for your support ❤️ |
331 |
| -
|
332 |
| -## 👥 Authors |
333 |
| -
|
334 |
| -- [@Anuj Khandelwal](https://www.github.com/anujonthemove) |
335 |
| -- [@Prakhar Agarwal](https://github.com/prakhar19) |
0 commit comments