Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increased version #11

Open
wants to merge 74 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
0c56c87
init
cshlopes May 10, 2018
74393c2
adding new rules
riquellopes May 10, 2018
731a55e
creating simple tasks
riquellopes May 10, 2018
f0c3f3c
pep validation
riquellopes May 10, 2018
eff9c90
starting refactoring
riquellopes May 10, 2018
13adecc
creating the first test for get_truncates
riquellopes May 10, 2018
6875784
fixing a bug
riquellopes May 10, 2018
fe1a693
fixing a bug
riquellopes May 10, 2018
97d4bdb
writing a new test for sql delete
riquellopes May 10, 2018
e8c6c91
covering def listify
riquellopes May 11, 2018
7c0dc6e
updating the default domain and creating a simple test
riquellopes May 11, 2018
32a642c
adding a comment
riquellopes May 11, 2018
adcf6da
Refactoring
cshlopes May 11, 2018
15ac475
adding ci support
cshlopes May 11, 2018
ca93a06
adding the status ci image
cshlopes May 11, 2018
21775a7
fix the install command
cshlopes May 11, 2018
97a7779
changing import package at top
cshlopes May 11, 2018
f42a92a
refactoring
riquellopes May 11, 2018
1093e9b
refactoring
riquellopes May 11, 2018
1ffc9fb
adding support to dynamic primary key
riquellopes May 12, 2018
e29e989
setting the log on project
riquellopes May 13, 2018
85f29ff
refactoring code to improve the tests
riquellopes May 13, 2018
ec3d17d
breaking de code into small parts
riquellopes May 13, 2018
0a86f30
adding new random fields
riquellopes May 13, 2018
92c4b55
adding ipdb
riquellopes May 14, 2018
253f7ae
copy sample files to package
riquellopes May 14, 2018
8164d2b
refactoring code
riquellopes May 14, 2018
6feccff
updating the documentation and add requires at setup
riquellopes May 14, 2018
0a10e1f
adding exception support for sql
cshlopes May 14, 2018
0687936
updating readme
cshlopes May 14, 2018
fd9e076
updating readme
cshlopes May 14, 2018
5595782
updating readme
cshlopes May 14, 2018
93bd9c0
updating readme
cshlopes May 14, 2018
03162ba
updating readme
cshlopes May 15, 2018
a4246d6
fixing path of file
riquellopes May 16, 2018
ce0ab71
Inclusão de informação sobre o CNPJ
ronaldosantoscs Jun 20, 2018
1c63e53
Inclusion of CNPJ randomization
ronaldosantoscs Jun 21, 2018
e79dc9d
Merge pull request #1 from ronaldorts/master
riquellopes Jun 21, 2018
3443c63
adding support to coverall
cshlopes Jun 28, 2018
6c9600b
adding the coveralls badge
cshlopes Jun 28, 2018
d5c5fb7
adding the coveralls badge
cshlopes Jun 28, 2018
6480ac4
Update README.md
riquellopes Jun 28, 2018
d1bd8c7
changing version of pytest
cshlopes Jun 28, 2018
d3245f3
Merge branch 'master' of github.com:riquellopes/mysql-anonymous
cshlopes Jun 28, 2018
5cee78b
deleting coveralls file
riquellopes Jun 28, 2018
e68bf27
adding the extra commands for coverage
riquellopes Jun 28, 2018
f97c118
omit package tests
riquellopes Jun 28, 2018
c5e3671
creating a new task
riquellopes Jun 28, 2018
a4cdd3c
updating travis file
riquellopes Jun 29, 2018
b8c2d08
updating travis file
riquellopes Jun 29, 2018
33ea5e5
fix make
riquellopes Jun 29, 2018
a13a5f9
pep
riquellopes Oct 10, 2018
a15868a
pep
riquellopes Oct 10, 2018
60a0a14
adding support to Lorem Ipsum text field
riquellopes Oct 10, 2018
daebf03
removing empty test
riquellopes Oct 10, 2018
2119a8e
updating the readme
riquellopes Oct 10, 2018
758d3d5
adding absolute_import package
riquellopes Oct 10, 2018
5055ae7
upgrade the module to use items def
riquellopes Oct 11, 2018
69a8b2d
creating new tests
riquellopes Oct 26, 2018
6220213
Merge pull request #4 from riquellopes/feature/lorem-ipsum
riquellopes Oct 28, 2018
bb712af
manual merge
riquellopes Oct 28, 2018
2972a9d
adding a silent task
riquellopes Oct 28, 2018
a5d70e0
Merge pull request #7 from riquellopes/feature/geral
riquellopes Oct 28, 2018
89b0102
Merge remote-tracking branch 'origin/master' into feature/tests
riquellopes Oct 28, 2018
a4bab5c
Merge pull request #6 from riquellopes/feature/tests
riquellopes Oct 28, 2018
6034d7e
fix the version of pyyaml
riquellopes Mar 5, 2019
d736e08
adding filter
riquellopes Mar 5, 2019
2ba2b07
removing filter
riquellopes Mar 5, 2019
ea522b6
changing pyyaml version
riquellopes Mar 5, 2019
6046fd1
package secure
riquellopes Mar 11, 2019
6d5bcc0
Update README.md
riquellopes Feb 3, 2022
7dad104
incresed version
riquellopes Feb 4, 2022
17d9113
call class Loader
riquellopes Feb 4, 2022
8b5e8bd
added support to new python versions
riquellopes Feb 4, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[run]
omit =
*/tests*
107 changes: 106 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,107 @@
anon.sql
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# pyenv
.python-version

# celery beat schedule file
celerybeat-schedule

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/

# dump
*.sql
13 changes: 13 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
language: python

python:
- "3.5"
- "3.6"
- "3.9"
- "3.10"

install: "make setup"

script:
- make test
- make coveralls
25 changes: 25 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
.SILENT:
PIP=.venv/bin/pip
PYTEST=.venv/bin/pytest
PYTHON=.venv/bin/python
COVERALLS=.venv/bin/coveralls


test:clean
PYTHONPATH=anonymize ${PYTEST} -s -v --cov=anonymize --cov-report term-missing tests/${path}

venv:
virtualenv .venv --python=python3

setup:venv
${PIP} install -U pip
${PIP} install -r requirements_dev.txt

clean:
find . -name "*.pyc" -exec rm -rf {} \;

sample:
PYTHONPATH=anonymize ${PYTHON} anonymize/__init__.py

coveralls:
${COVERALLS}
54 changes: 49 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,69 @@
## Mysql Anonymous

[![Build Status](https://travis-ci.org/riquellopes/mysql-anonymous.svg?branch=master)](https://travis-ci.org/riquellopes/mysql-anonymous)
[![Coverage Status](https://coveralls.io/repos/github/riquellopes/mysql-anonymous/badge.svg?branch=master)](https://coveralls.io/github/riquellopes/mysql-anonymous?branch=master)

Contributors can benefit from having real data when they are
developing. This script can do a few things (see `anonymize.yml`):
developing. This script can do a few things (see `sample1.yml` or `sample2.yml`):

* Truncate any tables (logs, and other cruft which may have sensitive data)
* Nullify fields (emails, passwords, etc)
* Fill in random/arbitrary data:
* Random integers
* Random IP addresses
* Random Cell Phone
* Random Phone
* Random [CPF](https://pt.wikipedia.org/wiki/Cadastro_de_pessoas_f%C3%ADsicas)
* Random [CNPJ](https://pt.wikipedia.org/wiki/Cadastro_Nacional_da_Pessoa_Jur%C3%ADdica)
* Email addresses
* Usernames
* Text [Loren Ipsum](https://www.lipsum.com/)

* Delete rows based on simple rules: e.g.
``DELETE FROM mytable WHERE private = "Yes"``:

```yml
database:
tables:
mytable:
delete:
nullify:
private: Yes
```

* Apply rules exception in some cases: e.g.
``UPDATE mytable SET cellphone=NULL WHERE id NOT IN(556, 889)``:

```yml
database:
tables:
mytable:
exception:
- 556
- 889
nullify:
- cellphone
```

* Define an other name for primary key of table: e.g.
``UPDATE mytable SET `email` = CONCAT(user_id, '@example.com')``:

### Usage
```yml
database:
tables:
primary_key: user_id
mytable:
random_email: email
```

python anonymize.py > anon.sql
cat anon.sql | mysql
Installation
------------
```sh
pip install https://github.com/riquellopes/mysql-anonymous/tarball/master
```
CookBook
--------
```sh
anonymize --sample-one
anonymize --sample-two
anonymize -y database.yml
```
119 changes: 0 additions & 119 deletions anonymize.py

This file was deleted.

Loading