Skip to content

GitLab

Commit 785ce268 authored by Mark Antony's avatar Mark Antony
Browse files

adding the index file

parent 7a91657a
Pipeline #108104 passed with stage
in 1 minute and 35 seconds
Hide whitespace changes
Inline Side-by-side
README.md
View file @ 785ce268
![Build Status](https://gitlab.com/pages/gitbook/badges/master/build.svg)
# Datateam Documentation
---
Technical documentation relating to everything technical
Example [GitBook] website using GitLab Pages.
## Gitlab related
Learn more about GitLab Pages at https://pages.gitlab.io and the official
documentation https://docs.gitlab.com/ce/user/project/pages/.
[How to set up the Project structure](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/1.1-How-to-set-up-Gitlab-Repository-structure-in-Datateam)
---
## Cloud releated
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*
### AWS
- [GitLab CI](#gitlab-ci)
- [Building locally](#building-locally)
- [GitLab User or Group Pages](#gitlab-user-or-group-pages)
- [Did you fork this project?](#did-you-fork-this-project)
- [Troubleshooting](#troubleshooting)
really useful [AWS CLI commands ](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/5.1-Useful-AWS-CLI-commands)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
[AWS Rules](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/project_documentation/AWS_rules.md)
## GitLab CI
### Azure
This project's static Pages are built by [GitLab CI][ci], following the steps
defined in [`.gitlab-ci.yml`](.gitlab-ci.yml):
## Cost related
```yaml
# requiring the environment of NodeJS 8.9.x LTS (carbon)
image: node:8.9
- [Cost saving](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/3.0-Cost-Saving)
- [Tagging resources](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/3.1-Tagging-Cloud-Resources)
- [Billing alarms](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/3.2-Billing-Alarms-via-AWS-budgets)
- [Creating Billing alarms with Cloudwatch](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/3.3-Creating-a-Billing-Alarm-using-Amazon-CloudWatch)
# add 'node_modules' to cache for speeding up builds
cache:
paths:
- node_modules/ # Node modules and dependencies
## Architecture related - How to write an application architecture
- [Rational Unified process](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/2.0-Rational-Unified-process-(RUP))
- [Use Case Views](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/2.1-Use-case-View)
- [Logical View ](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/2.2-Logical-View)
- [Logical view example - Bank ATM](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/2.2.1-Logical-View-Example-for-an-ATM-System)
- [Data science application example](url) In progress
- [Process, Implementation and Deployment View ](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/2.3-The-Process,-Implementation-and-Deployment-Views)
- [Views example - Bank ATM ](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/2.3.1-Views-for-the-ATM-example)
- [Data science application example](url) In Progress
- [AWS Diagram examples](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/2.3.2-Architecture-diagrams-for-AWS)
- [Azure Diagram examples](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/2.3.3-Architecture-Diagrams-for-Azure)
before_script:
- npm install gitbook-cli -g # install gitbook
- gitbook fetch latest # fetch latest stable version
- gitbook install # add any requested plugins in book.json
#- gitbook fetch pre # fetch latest pre-release version
#- gitbook fetch 2.6.7 # fetch specific version
### Swagger related
# the 'pages' job will deploy and build your site to the 'public' path
pages:
stage: deploy
script:
- gitbook build . public # build to public path
artifacts:
paths:
- public
only:
- master # this job will affect only the 'master' branch
```
To help define the API we recommend Swagger
## Building locally
- [Swagger API specification](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/2.4.1-Swagger-Open-API-Specification)
- [Swagger tool](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/2.4.2-The-Swagger-Editor)
To work locally with this project, you'll have to follow the steps below:
1. Fork, clone or download this project
1. [Install][] GitBook `npm install gitbook-cli -g`
1. Fetch GitBook's latest stable version `gitbook fetch latest`
1. Preview your project: `gitbook serve`
1. Add content
1. Generate the website: `gitbook build` (optional)
1. Push your changes to the master branch: `git push`
## Development Related
Read more at GitBook's [documentation][].
### Python
## GitLab User or Group Pages
All things [python](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/4.0-Development-with-Python)
To use this project as your user/group website, you will need one additional
step: just rename your project to `namespace.gitlab.io`, where `namespace` is
your `username` or `groupname`. This can be done by navigating to your
project's **Settings**.
### How to create Python Lambdas
Read more about [user/group Pages][userpages] and [project Pages][projpages].
## Did you fork this project?
If you forked this project for your own use, please go to your project's
**Settings** and remove the forking relationship, which won't be necessary
unless you want to contribute back to the upstream project.
### Python Cheetsheets
## Troubleshooting
* [Beginners Python](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/Beginners-Python-Cheat-Sheet.pdf)
* [Intermediate Python](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/Intermediate-Python-Cheat-Sheet.pdf)
* [Bokeh](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/bokeh.pdf)
* [Importing data](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/import_data.pdf)
* [Jupyter](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/jupyter.pdf)
* [Keras](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/keras.pdf)
* [Matplotlib](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/matplotlib.pdf)
* [Numpy Basics](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/numpy_basics.pdf)
* [Pandas](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/pandas.pdf)
* [Pandas Basics](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/pandas_basics.pdf)
* [pyspark df](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/pyspark_df.pdf)
* [pyspark rdd](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/pyspark_rdd.pdf)
* [python regexp](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/python-regular-expressions-cheat-sheet.pdf)
* [scikit-learn](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/scikit-learn.pdf)
* [SciPy](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/scipy_linear_algebra.pdf)
* [Seaborn](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Python/seaborn.pdf)
1. CSS is missing! That means two things:
### R cheatsheets
Either that you have wrongly set up the CSS URL in your templates, or
your static generator has a configuration option that needs to be explicitly
set in order to serve static assets under a relative URL.
* [Data visualisation in R](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/R/Comprehensive%20Guide%20to%20Data%20Visualization_R.pdf)
* [data table](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/R/data_table.pdf)
* [tidiverse beginners](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/R/tidiverse_beginners.pdf)
* [xts](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/R/xts.pdf)
----
### Neo4J Documentation
* [Neo4j Cypher Manual](https://neo4j.com/docs/cypher-manual/4.0/)
* [Neo4j Cypher Refcard](https://neo4j.com/docs/cypher-refcard/current/)
* [APOC Documentation](https://neo4j.com/docs/labs/apoc/current/)
* [Neo4j Drivers Manual](https://neo4j.com/docs/driver-manual/current/)
* [Neo4j Community Site](https://community.neo4j.com/c/general/online-training)
### General documentation
* [Apache Airflow](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Markdown_files/airflow.md)
* [Docker](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Markdown_files/docker.md)
* [Hadoop](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Markdown_files/hadoop.md)
* [Hbase](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Markdown_files/hbase.md)
* [Hive](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Markdown_files/hive.md)
* [Kafka](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Markdown_files/kafka.md)
* [MariaDB](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Markdown_files/mariadb.md)
* [Pig](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Markdown_files/pig.md)
* [Redis](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Markdown_files/redis.md)
* [Spark](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/Datascience-stuff/Markdown_files/spark.md)
### Bash and Command line tools
- How to use [curl](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/6.1-How-to-use-curl)
- Bash Tips and [Tricks](https://git.fpfis.eu/datateam/datateam-documentation/-/wikis/6.2-Bash-Tricks-and-Tips)
## Security related
- [NIST document used for DIGIT Security risk assessment](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/project_documentation/NIST.SP.800-53r5-draft.pdf)
- [Useful tool to convert files](https://git.fpfis.eu/datateam/DevSecOps/cyberchef-tool/-/blob/master/CyberChef_v9.28.0.html)
- [How to check how secure your browser is](https://clienttest.ssllabs.com:8443/ssltest/viewMyClient.html)
- [Get your IP Address](https://ifconfig.co/)
- [A load of free how to´s security books](https://github.com/Hack-with-Github/Free-Security-eBooks)
- [AWS security](https://wellarchitectedlabs.com/security/)
- [check if your browser is protected against tracking ](https://panopticlick.eff.org/)
- [check what private information your browser is leaking](https://privacy.net/analyzer/)
- [check your website for insecure pages](https://www.jitbit.com/sslcheck/)
## Free technical books
- [Docker workshop](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/THE_DOCKER_WORKSHOP.pdf)
- [SQL Workshop](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/THE_SQL_WORKSHOP.pdf)
- [Professional SQL](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/SQLNotesForProfessionals.pdf)
- [Algorithms How to ](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/AlgorithmsNotesForProfessionals.pdf)
- [Bash for Professionals](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/BashNotesForProfessionals.pdf)
- [Everything Curl](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/Everything_curl.pdf)
- [SSH book](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/SSH__The_Secure_Shell__2nd_Edition.pdf)
- [Learn Power BI (free book from Packt)](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/LEARN_POWER_BI.pdf)
- [Linux for professionals](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/LinuxNotesForProfessionals.pdf)
- [Mongo DB for professionals](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/MongoDBNotesForProfessionals.pdf)
- [MySQL for Professionals](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/MySQLNotesForProfessionals.pdf)
- [Oracle for Professionals](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/OracleDatabaseNotesForProfessionals.pdf)
- [PostGres for professionals](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/PostgreSQLNotesForProfessionals.pdf)
- [Python for Professionals](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/PythonNotesForProfessionals.pdf)
- [R for Professionals](https://git.fpfis.eu/datateam/datateam-documentation/-/blob/master/reference_documentation/RNotesForProfessionals.pdf)
Forked from @virtuacreative
[ci]: https://about.gitlab.com/gitlab-ci/
[GitBook]: https://www.gitbook.com/
[host the book]: https://gitlab.com/pages/gitbook/tree/pages
[install]: http://toolchain.gitbook.com/setup.html
[documentation]: http://toolchain.gitbook.com
[userpages]: https://docs.gitlab.com/ce/user/project/pages/introduction.html#user-or-group-pages
[projpages]: https://docs.gitlab.com/ce/user/project/pages/introduction.html#project-pages
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment