diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md new file mode 100755 index 0000000..fad9895 --- /dev/null +++ b/.github/ISSUE_TEMPLATE.md @@ -0,0 +1,19 @@ +## Problem/Motivation + +> (Why the issue was filed) + +## Expected behavior + +> (What you expected to happen) + +## Actual behavior + +> (What actually happened) + +## Steps to reproduce + +> (How can someone else make/see it happen) + +## Proposed changes + +> (If you have a proposed change, workaround or fix, describe the rationale behind it) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100755 index 0000000..57fcd54 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,7 @@ +## Proposed Changes + +> (Describe the changes and rationale behind them) + +## Related Issues + +> ([Github link](https://help.github.com/articles/autolinked-references-and-urls/) to related issues or pull requests) diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..4d41055 --- /dev/null +++ b/.gitignore @@ -0,0 +1,13 @@ +__pycache__/ +*.py[cod] +build/ +dist/ +*.egg-info/ +.tox/ +bin/ +pip/ +lib/ +include/ +install/ +.Python +pip-selfcheck.json \ No newline at end of file diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 0000000..dc3b428 --- /dev/null +++ b/.travis.yml @@ -0,0 +1,20 @@ +language: python + +python: + - "2.6" + - "2.7" + - "3.2" + - "3.3" + - "3.4" + - "3.5" + - "3.6" + - "pypy" + - "pypy3" + +cache: + - pip + +install: + - pip install tox-travis +script: + - tox diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..64899eb --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,71 @@ +# Code of conduct + +## Our pledge + +In the interest of fostering an open and welcoming environment, we as +contributors and maintainers pledge to making participation in our project and +our community a harassment-free experience for everyone, regardless of age, body +size, disability, ethnicity, gender identity and expression, level of experience, +nationality, personal appearance, race, religion, or sexual identity and +orientation. + +## Our standards + +Examples of behavior that contributes to creating a positive environment +include: + +- Using welcoming and inclusive language +- Being respectful of differing viewpoints and experiences +- Gracefully accepting constructive criticism +- Focusing on what is best for the community +- Showing empathy towards other community members + +Examples of unacceptable behavior by participants include: + +- The use of sexualized language or imagery and unwelcome sexual attention or advances +- Trolling, insulting/derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others' private information, such as a physical or electronic address, without explicit permission +- Other conduct which could reasonably be considered inappropriate in a professional setting + +## Our responsibilities + +Project maintainers are responsible for clarifying the standards of acceptable +behavior and are expected to take appropriate and fair corrective action in +response to any instances of unacceptable behavior. + +Project maintainers have the right and responsibility to remove, edit, or +reject comments, commits, code, wiki edits, issues, and other contributions +that are not aligned to this Code of Conduct, or to ban temporarily or +permanently any contributor for other behaviors that they deem inappropriate, +threatening, offensive, or harmful. + +## Scope + +This Code of Conduct applies both within project spaces and in public spaces +when an individual is representing the project or its community. Examples of +representing a project or community include using an official project e-mail +address, posting via an official social media account, or acting as an appointed +representative at an online or offline event. Representation of a project may be +further defined and clarified by project maintainers. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported by contacting the project lead at frenck@geekchimp.com. All +complaints will be reviewed and investigated and will result in a response that +is deemed necessary and appropriate to the circumstances. The project lead is +obligated to maintain confidentiality with regard to the reporter of an incident. +Further details of specific enforcement policies may be posted separately. + +Project maintainers who do not follow or enforce the Code of Conduct in good +faith may face temporary or permanent repercussions as determined by other +members of the project's leadership. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], +version 1.4, available at [http://contributor-covenant.org/version/1/4][version] + +[homepage]: http://contributor-covenant.org +[version]: http://contributor-covenant.org/version/1/4/ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..608b885 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,29 @@ +# Contributing + +When contributing to this repository, please first discuss the change you wish +to make via issue, email, or any other method with the owners of this repository +before making a change. + +Please note we have a code of conduct, please follow it in all your interactions +with the project. + +## Issues and feature requests + +You've found a bug in the source code, a mistake in the documentation or maybe +you'd like a new feature? You can help us by submitting an issue to our +[GitHub Repository][github]. Before you create an issue, make sure you search +the archive, maybe your question was already answered. + +Even better: You could submit a pull request with a fix / new feature! + +## Pull request process + +1. Search our repository for open or closed [pull requests][prs] that relates + to your submission. You don't want to duplicate effort. + +2. You may merge the pull request in once you have the sign-off of two other + developers, or if you do not have permission to do that, you may request + the second reviewer to merge it for you. + +[github]: https://github.com/hassio-addons/docker-context-streamer/issues +[prs]: https://github.com/hassio-addons/docker-context-streamer/pulls \ No newline at end of file diff --git a/LICENSE.md b/LICENSE.md new file mode 100644 index 0000000..e8635ed --- /dev/null +++ b/LICENSE.md @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2017 Franck Nijhof + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md new file mode 100644 index 0000000..6f0a550 --- /dev/null +++ b/README.md @@ -0,0 +1,88 @@ +# Docker Context Streamer + +![Project Stage][project-stage-shield] +![Maintenance][maintenance-shield] +![Awesome][awesome-shield] +[![License][license-shield]](LICENSE.md) + +This little tool allows you to add file context to Dockerfiles fed to +Docker using STDIN by creating a tar file stream. + +## What is does + +If you, for whatever reason, are streaming your Dockerfile into the Docker +build command, your file context will be ignored by Docker. e.g., Stuff like +ADD and COPY won't work. This tool work around this problem by using Dockers +ability to accept a tar stream containing the full context. + +Pipe your in memory Dockerfile into the Docker Context Streamer, +specify the directory on disk you'd like to use as the file context, and next +stream the output of this tool into Docker build. + +## Example + +```bash +$ export DOCKERFILE="FROM alpine" +# Do some stuff with your Docker file here, like sed replacing or adding things +$ echo $DOCKERFILE | docker-context-streamer ./context/dir | docker build - +``` + +## Notice + +Using this tool is a general warning sign that you are doing it wrong. +This tool is created for a very special case, where there was no other way. + +## Installation + +Using pip, the Python package manager: + +```bash +sudo pip install docker-context-streamer +``` + +## Contributing + +This is an active open-source project. We are always open to people who want to +use the code or contribute to it. + +We've set up a separate document for our [contribution guidelines](CONTRIBUTING.md). + +Thank you for being involved! :heart_eyes: + +## Authors & contributors + +The original setup of this repository is by [Franck Nijhof][frenck]. + +For a full list of all authors and contributors, +check [the contributor's page][contributors]. + +## License + +MIT License + +Copyright (c) 2017 Franck Nijhof + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +[awesome-shield]: https://img.shields.io/badge/awesome%3F-yes-brightgreen.svg +[contributors]: https://github.com/hassio-addons/docker-context-streamer/graphs/contributors +[frenck]: https://github.com/frenck +[license-shield]: https://img.shields.io/github/license/hassio-addons/docker-context-streamer.svg +[maintenance-shield]: https://img.shields.io/maintenance/yes/2017.svg +[project-stage-shield]: https://img.shields.io/badge/Project%20Stage-Experimental-yellow.svg diff --git a/dcs/__init__.py b/dcs/__init__.py new file mode 100644 index 0000000..b4323c7 --- /dev/null +++ b/dcs/__init__.py @@ -0,0 +1,41 @@ +# -*- coding: utf-8 -*- +# +# MIT License +# +# Copyright (c) 2017 Franck Nijhof +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +""" +Docker Context Streamer. + +This little tool allows you to add file context to Dockerfiles fed to +Docker using STDIN by creating a tar file stream. +""" + + +APP_NAME = 'docker-context-streamer' +APP_VERSION = '0.0.1' +APP_DESCRIPTION = __doc__ + +__author__ = u'Franck Nijhof' +__copyright__ = u'Copyright 2017, Franck Nijhof' +__license__ = 'GPLv3' +__url__ = 'https://github.com/hassio-addons/docker-context-streamer' +__version__ = APP_VERSION diff --git a/dcs/__main__.py b/dcs/__main__.py new file mode 100644 index 0000000..67f6953 --- /dev/null +++ b/dcs/__main__.py @@ -0,0 +1,29 @@ +# -*- coding: utf-8 -*- +# +# MIT License +# +# Copyright (c) 2017 Franck Nijhof +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +"""Main entry point.""" +from dcs.cli import run + +if __name__ == '__main__': + run() diff --git a/dcs/cli.py b/dcs/cli.py new file mode 100644 index 0000000..cb4e5ec --- /dev/null +++ b/dcs/cli.py @@ -0,0 +1,79 @@ + +# -*- coding: utf-8 -*- +# +# MIT License +# +# Copyright (c) 2017 Franck Nijhof +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +"""CLI Handler for the Docker Context Streamer. + +This file handles the parsing of the CLI input +""" + +import argparse +import os.path +import sys + +from dcs import APP_DESCRIPTION, APP_NAME, APP_VERSION +from dcs.streamer import Streamer + + +def run(argv=None): + """Run the Docker Context Streamer.""" + parser = argparse.ArgumentParser( + prog=APP_NAME, + description=APP_DESCRIPTION, + epilog=("Example: cat ~/Dockerfile | " + "docker-context-streamer . | docker build -") + ) + parser.add_argument( + 'context', + type=str, + default='.', + help='Path to directory to use as context' + ) + parser.add_argument( + '-v', '--version', + action='version', + version='%s %s' % (APP_NAME, APP_VERSION) + ) + + parsed = parser.parse_args() + + if sys.stdin.isatty(): + parser.error("Missing Dockerfile on STDIN") + + dockerfile = sys.stdin.read() + dockerignore = "" + dockerignorefile = os.path.relpath( + os.path.join( + parsed.context, + '.dockerignore' + ) + ) + if os.path.isfile(dockerignorefile): + with open(dockerignorefile, 'r') as f: + dockerignore = (f.read()).split('\n') + + streamer = Streamer( + dockerfile, os.path.relpath(parsed.context), dockerignore + ) + streamer.stream_context() diff --git a/dcs/streamer.py b/dcs/streamer.py new file mode 100644 index 0000000..17d870e --- /dev/null +++ b/dcs/streamer.py @@ -0,0 +1,79 @@ +# -*- coding: utf-8 -*- +# +# MIT License +# +# Copyright (c) 2017 Franck Nijhof +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +"""Docker Context Streamer. + +This provides the actual Docker Context Streamer. +It will in take the given Dockerfile (via STDIN) and a path, and will +output a tar stream containing both. +""" + +from fnmatch import fnmatchcase +from six import StringIO +from tarfile import TarFile, TarInfo +import sys + + +class Streamer(object): + """Docker Context Streamer. + + This provides the actual Docker Context Streamer. + It will in take the given Dockerfile (via STDIN) and a path, and will + output a tar stream containing both. + """ + + def __init__(self, dockerfile, context, dockerignore): + """Class init.""" + self.dockerfile = dockerfile + self.context = context + self.dockerignore = dockerignore + + def exclude(self, filename): + """Exclude files from the stream.""" + if filename.startswith(self.context): + filename = filename[len(self.context)+1:] + + if filename == 'Dockerfile' or filename == '.dockerignore': + return True + elif any( + fnmatchcase(filename, pattern) for pattern in self.dockerignore + ): + return True + else: + print filename + return False + + def stream_context(self): + """Start streaming the tar context for Docker.""" + with TarFile.open(mode='w|', fileobj=sys.stdout) as tarfile: + + tarfile.add( + self.context, + arcname='.', + exclude=self.exclude + ) + + tarinfo = TarInfo('./Dockerfile') + tarinfo.size = len(self.dockerfile) + tarfile.addfile(tarinfo, StringIO(self.dockerfile)) diff --git a/setup.cfg b/setup.cfg new file mode 100644 index 0000000..9d08805 --- /dev/null +++ b/setup.cfg @@ -0,0 +1,5 @@ +[bdist_wheel] +universal=1 + +[metadata] +description-file = README.md \ No newline at end of file diff --git a/setup.py b/setup.py new file mode 100644 index 0000000..469cfc9 --- /dev/null +++ b/setup.py @@ -0,0 +1,38 @@ +# -*- coding: utf-8 -*- + +from setuptools import setup, find_packages + +from dcs import ( + __author__, __license__, __url__, + APP_NAME, APP_VERSION, APP_DESCRIPTION +) + +setup( + name=APP_NAME, + version=APP_VERSION, + author=__author__, + description=APP_DESCRIPTION.split('\n')[0], + long_description=APP_DESCRIPTION, + license=__license__, + url=__url__, + keywords=['docker', 'dockerfile', 'stream'], + classifiers=[ + 'Development Status :: 3 - Alpha', + 'Environment :: Console', + 'Intended Audience :: Developer', + 'License :: OSI Approved :: MIT Licens' + 'Programming Language :: Python :: 2' + 'Programming Language :: Python :: 3' + 'Topic :: Software Development :: Build Tools', + 'Topic :: System', + 'Topic :: Utilities', + ], + packages=find_packages(), + requires=["six"], + install_requires=["six >=1.10"], + tests_require=['tox'], + entry_points={'console_scripts': [ + 'docker-context-streamer=dcs.cli:run', + 'dcs=dcs.cli:run' + ]} +) \ No newline at end of file diff --git a/test/Dockerfile b/test/Dockerfile new file mode 100644 index 0000000..ebe2e37 --- /dev/null +++ b/test/Dockerfile @@ -0,0 +1 @@ +NOOOO \ No newline at end of file diff --git a/test/context/.dockerignore b/test/context/.dockerignore new file mode 100644 index 0000000..3ba2047 --- /dev/null +++ b/test/context/.dockerignore @@ -0,0 +1,2 @@ +ignored.* +.git \ No newline at end of file diff --git a/test/context/Dockerfile b/test/context/Dockerfile new file mode 100644 index 0000000..6d9dbd7 --- /dev/null +++ b/test/context/Dockerfile @@ -0,0 +1,2 @@ +# I must be ignored... +FROM alpine diff --git a/test/context/etc/config.conf b/test/context/etc/config.conf new file mode 100644 index 0000000..d2d7350 --- /dev/null +++ b/test/context/etc/config.conf @@ -0,0 +1 @@ +# Fake config file for testing \ No newline at end of file diff --git a/test/context/ignored.txt b/test/context/ignored.txt new file mode 100644 index 0000000..73064c6 --- /dev/null +++ b/test/context/ignored.txt @@ -0,0 +1 @@ +This file is for testing .dockerignore \ No newline at end of file diff --git a/test/context/testfile.txt b/test/context/testfile.txt new file mode 100644 index 0000000..e8bcfb2 --- /dev/null +++ b/test/context/testfile.txt @@ -0,0 +1,2 @@ +HI! +I'm a test file :) \ No newline at end of file diff --git a/tox.ini b/tox.ini new file mode 100644 index 0000000..4337237 --- /dev/null +++ b/tox.ini @@ -0,0 +1,31 @@ +[tox] +envlist = pep8, pep257, flake8, py27, py34, py35, py36, tar +skip_missing_interpreters = True +whitelist_externals = + cat + echo + tar + +[testenv] +commands = + pip install . + python --version + +[testenv:flake8] +commands = + pip install . + flake8 --max-complexity 10 dcs +deps = + flake8 + +[testenv:pep257] +commands = + pip install . + pep257 dcs +deps = + pep257 + +[testenv:tar] +commands = + pip install . + echo "FROM ..." | ./bin/docker-context-streamer ./test/context | tar -t \ No newline at end of file