Installation ============ Debian GNU/Linux ---------------- .. note:: If there has not been much time since the release, the Debian packages for the latest version may not yet be available. PATHspider is packaged for Debian and packages are made available for the testing and stable-backports distributions. If you are running Debian stable, ensure that you have `enabled the stable-backports repository `_ in your apt sources. To install PATHspider, simply run: .. code-block:: shell sudo apt install pathspider Vagrant ------- .. warning:: Depending on the set up of your Vagrant virtualization provider, some tests may be affected. It is wise to test against known configurations to ensure that your networking set up has a clear path to the Internet before running larger measurement campaigns. On systems other than Linux systems, you may use `Vagrant `_ to run PATHspider. This may also be useful during development. A Vagrantfile is provided that will create a Debian-based virtual machine with all the PATHspider dependencies installed. In the virtual machine, the PATHspider code will be copied to ``/vagrant``. To improve compatibility across platforms, this is not synchronized with the repository outside of the Vagrant image. Expert users may edit the ``Vagrantfile`` to achieve this. PATHspider is installed in development mode, meaning that this is also the location of the PATHspider code that will be run when running the ``/usr/bin/pspdr`` command inside the virtual machine. Assuming that you have Vagrant and a virtualisation provider (e.g. VirtualBox) installed, you can get started with: .. code-block:: shell vagrant up vagrant ssh Depending on the speed of your Internet connection, this may take a long time. virtualenv ---------- .. warning:: PATHspider 2.0 depends on pycurl >= 7.43.0.1, released on the 7th December 2017. If you have errors when running PATHspider similar to ``AttributeError: module 'pycurl' has no attribute 'CONNECT_TO'`` then it is most likely the case that your version of pycurl is too old. `virtualenv` is a tool to create isolated Python environments. This allows you to install the dependencies necessary for PATHspider without having them conflict with your system libraries used for other applications on the system. The following instructions assume a Debian GNU/Linux system and may have to be modified on other systems: .. code-block:: shell sudo apt install libtrace-dev libldns-dev python3-dev python3-virtualenv mkdir ~/psenv && cd ~/psenv python3 -m virtualenv -p /usr/bin/python3 . source bin/activate export PATH=$PWD/bin:$PATH git clone https://github.com/nevil-brownlee/python-libtrace.git pushd python-libtrace && python3 setup.py install && popd git clone https://github.com/mami-project/pathspider.git pushd pathspider && \ pip install -r requirements.txt && \ pip install -r requirements_dev.txt && \ python3 setup.py develop && popd pspdr test Ensure that all tests have passed before beginning to measure or develop with PATHspider. To re-enter the virtual environment from another shell session: .. code-block:: shell cd ~/psenv source bin/activate export PATH=$PWD/bin:$PATH pspdr test Source ------ .. warning:: PATHspider 2.0 depends on pycurl >= 7.43.0.1, released on the 7th December 2017. If you have errors when running PATHspider similar to ``AttributeError: module 'pycurl' has no attribute 'CONNECT_TO'`` then it is most likely the case that your version of pycurl is too old. If you are working from the source distribution (e.g. cloned git repository) then you will need to install the required dependencies. On Debian GNU/Linux, assuming you have the stable-backports repository enabled if you are running stable: .. code-block:: shell sudo apt build-dep pathspider .. note:: This will install both the runtime and the build dependencies required for PATHspider, its testsuite and its documentation. On other platforms, you may install most of the dependencies required via pip: .. code-block:: shell pip3 install -r requirements.txt Unfortunately, `python-libtrace `_ is not available on PyPI and so must be installed seperately. You will also need to ensure that for both pycurl and python-libtrace you have the build dependencies available as these are compiled CPython modules. If you wish to build the documentation from source or to use the testsuite, and you are installing your dependencies via pip, you will also need the following dependencies: .. code-block:: shell pip3 install -r requirements_dev.txt With the dependencies installed, you can install PATHspider with: .. code-block:: shell python3 setup.py install cloud-init ---------- The following cloud-config script installs PATHspider globally in the VM. The default user is `ubuntu`. Include your public key to ssh into the VM. Customise this to your needs. You may want to change the hostname. :: #cloud-config # Hostname management preserve_hostname: False hostname: spider fqdn: spider.local package_update: true package_upgrade: true ssh_authorized_keys: - ssh-rsa packages: - git - python3 - python3-pip - python3-setuptools - python3-pycurl - libtrace-dev - libldns-dev write_files: - content: | #!/bin/bash export LC_ALL=C # Select the stable pathspider release, comment for github clone REL=2.0.1 cd /tmp git clone https://github.com/nevil-brownlee/python-libtrace.git cd python-libtrace/ make install-py3 cd - if [ -z "$REL" ]; then git clone https://github.com/mami-project/pathspider else wget -q -O - https://github.com/mami-project/pathspider/archive/$REL.tar.gz | tar -xzvf - fi if [ -d pathspider ]; then cd pathspider # github clone else cd pathspider-$REL # release fi pip3 install -r requirements.txt python3 setup.py install cd - path: /root/build-psp.sh permissions: 0755 runcmd: - ls -al /root > /install-psp.log - /root/build-psp.sh # Configure where output will go output: all: ">> /var/log/cloud-init.log" # configure interaction with ssh server ssh_svcname: ssh ssh_deletekeys: True ssh_genkeytypes: ['rsa', 'ecdsa']