Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 02:01
    barracuda156 commented #3263
  • May 28 12:14
    Josue-T commented #3118
  • May 27 18:31
    gitgitwhat closed #3339
  • May 27 16:05
    abravalheri commented #3339
  • May 27 16:04
    abravalheri commented #3339
  • May 27 16:03
    abravalheri commented #3339
  • May 27 15:52
    gitgitwhat commented #3339
  • May 27 12:21
    abravalheri unlabeled #3333
  • May 27 12:21
    abravalheri unlabeled #3333
  • May 27 12:21
    abravalheri labeled #3333
  • May 27 12:17
    abravalheri commented #3333
  • May 27 10:13
    dekoma4u commented #3333
  • May 27 08:55
    abravalheri unlabeled #3339
  • May 27 08:55
    abravalheri unlabeled #3339
  • May 27 08:55
    abravalheri labeled #3339
  • May 27 08:54
    abravalheri commented #3339
  • May 26 19:31
    gitgitwhat opened #3339
  • May 26 19:31
    gitgitwhat labeled #3339
  • May 26 19:31
    gitgitwhat labeled #3339
  • May 26 15:56
    codeandfire commented #3335
Paul Ganssle
@pganssle
Did you try pinning setuptools_scm to an older version in your pyproject.toml? Seems like it is most likely to be a change in that package.
Setuptools has no dependencies.
sdraper-CS
@sdraper-CS
Yes - that was indeed the problem. Issue has been reported on setuptools_scm git. Thanks
drocsid
@drocsid
Hi yeah I'm not terribly well versed in python but inherited a broken python project. Inside the project I have a setup.py file that follows:
# -*- coding: utf-8 -*-

from setuptools import setup, find_packages

setup(
    name='int-airflow',
    version='1.0.270',
    license='MIT',
    packages=find_packages(exclude=('tests', 'tests.*', 'hql')),
    install_requires=[
        'apache-airflow>=1.7.0,<1.11.0'
    ],
    classifiers=[
        'Programming Language :: Python :: 2.7'
    ]
)
The project tells me to python setup.py build which fails fast and gives no errors
but when I try python setup.py test then it seems like it tries to grab the dependencies and fails
drocsid
@drocsid
Using /Users/colinwilliams/code/int-airflow/.eggs/pytzdata-2019.1-py2.7.egg
Traceback (most recent call last):
  File "setup.py", line 18, in <module>
    'Programming Language :: Python :: 2.7'
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/setuptools/command/test.py", line 217, in run
    installed_dists = self.install_dists(self.distribution)
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/setuptools/command/test.py", line 208, in install_dists
    ir_d = dist.fetch_build_eggs(dist.install_requires)
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/setuptools/dist.py", line 717, in fetch_build_eggs
    replace_conflicting=True,
  File "/Users/colinwilliams/code/int-airflow/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (tzlocal 2.0.0b2 (/Users/colinwilliams/code/int-airflow/.eggs/tzlocal-2.0.0b2-py2.7.egg), Requirement.parse('tzlocal<2.0.0.0,>=1.5.0.0'), set(['pendulum']))
So I have two issues. 1)The version conflict, and 2) python setup.py build doesn't install the deps , since I only get the install, etc with python setup.py test
drocsid
@drocsid
I'm not sure if the version conflict is caused by the classifiers line or just the airflow dependency itself...
Ian
@ionox0
Will python setup.py install install the dependencies of listed dependencies automatically?
Paul Ganssle
@pganssle
@ionox0 Do not use setup.py install, use pip install instead.
And yes, pip will install transitive dependencies automatically.
Ian
@ionox0
thank you, I also realized I was missing the extras part of the package that I was trying to install
toil vs toil[cwl]
drocsid
@drocsid

@ionox0 or channel I'm trying to use airflow[qds] from an inherited setup.py project but not sure how to specify. You mention something similar above. My setup.py setup() has

install_requires=[
        'apache-airflow>=1.7.0,<1.11.0',
        'jinja2>=2.10.0',
        'werkzeug>=0.15.0'

    ],

but I don't know how to add the qds module. Can anybody help? I've tried to search this out but am just getting more confused.

Ian
@ionox0
'apache-airflow[qds]>=1.7.0,<1.11.0’
does that work?
im no expert
drocsid
@drocsid
I didn't think it did let me try again
So I try that and the pip install -e . but then when I bring up airflow I get
line 32, in <module>
    from qds_sdk.qubole import Qubole
ImportError: No module named qds_sdk.qubole
Done.
so it allows the install but it looks like it doesn't let me load the qds module.
Ian
@ionox0
might need to use extras_require?
drocsid
@drocsid
I tried that also already
Ian
@ionox0
setup(
    name="Project-A",
    ...
    extras_require={
        'PDF':  ["ReportLab>=1.2", "RXP"],
        'reST': ["docutils>=0.3"],
    }
)
you could try with python setup.py install as well
drocsid
@drocsid
install_requires=[
        'apache-airflow>=1.7.0,<1.11.0',
        'jinja2>=2.10.0',
        'werkzeug>=0.15.0'

    ],
    extras_require=[
        'qds' : 'airflow[qds]',
        ]
above people are saying not so use python setup.py install but to use pip install instead.
Ian
@ionox0
could it be
    extras_require=[
        'apache-airflow' : ‘qds',
        ]
nah nvm
drocsid
@drocsid
From the extras_require description it sounds like that's for defining my own project modules
Ian
@ionox0
right right
it does look like its installing qds
but install is erroring
Ian
@ionox0
Maybe qds_sdk.qubole is also a requirement?
Try with qds-sdk also in install_requires?
drocsid
@drocsid
That looks like it was the issue. My bad. Thanks!
Ian
@ionox0
Cool, awesome
drocsid
@drocsid
Well it was able to initalize the database without exception but then when I started the webserver I got No module named qds_sdk.qubole again...
from the webserver ui
despite adding it to install_requrires
Ian
@ionox0
what does pip show qds-sdk say?
drocsid
@drocsid
pip show qds-sdk
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Name: qds-sdk
Version: 1.12.0
Summary: Python SDK for coding to the Qubole Data Service API
Home-page: https://github.com/qubole/qds-sdk-py
Author: Qubole
Author-email: dev@qubole.com
License: Apache License 2.0
Location: /private/tmp/id-core-test/id-correlation-offline/env/lib/python2.7/site-packages
Requires: requests, boto, inflection, six, urllib3
Required-by: idc-offline
Ian
@ionox0
are you using a virtualenv? if so is your webserver installed within / using that same virtualenv?
(or using conda)?
drocsid
@drocsid
Yeah I'm not sure what I did wrong that time but it does seem to be working on another attempt. Thanks for the help!
Ian
@ionox0
Np!
Thomas Robitaille
@astrofrog
Is there a document somewhere that describes that the plan for Python packaging is to move away from setuptools commands? I've read it in a few comments but I'd like to find a more official place where this is mentioned so that I can use this to convince the developer communities I'm part of to move in that direction.
Paul Ganssle
@pganssle
No.
It's true that you should, and almost certainly there is at least documentation that says not to use specific commands.