These are chat archives for django/django

15th
May 2017
Andrew Backer
@awbacker
May 15 2017 04:07
I have a little question about the ORM. For a particular query i'm making, I'd like to use the ORM to generate only the where clause, for safety and because its quite dynamic. The select/from parts are a bit too complicated. Does anyone know a way to do just this, or does it really boil downto manually constructing it again
Kyle Mulka
@mulka
May 15 2017 04:09
Wait? Why don't you want the ORM to generate the select and from?
Andrew Backer
@awbacker
May 15 2017 06:35
nope, not really
sorry, its lunch time here, back now. its going to be all sorts of counts sum/case/when and garbage like that ;)
Andrew Backer
@awbacker
May 15 2017 06:41
I can do the 'where' if necessary, but.. its gotten so nice relying on django's understanding of postgres operators on json, hstore, arrays, etc... and it makes it super easy to dynamically create the nested ors/ands whatever
hard to give up :)
:]
@elcolie
May 15 2017 07:05
Hi I want to search Booking instance with range like in the given picture. How do you use the django-filter?
range-search.jpg
Booking properties are
    # first slot date for this booking
    start_date = models.DateField(verbose_name=_("start date"), null=True, blank=True)

    # last slot date for this booking
    end_date = models.DateField(verbose_name=_("end date"), null=True, blank=True)
S1, S2, S3 and S4 key search is overlap with given start_date, and end_date
Roel
@roelzkie15
May 15 2017 08:16

@elcolie hi you can use the range method

Booking.objects.filter(created_at__range=(start_date, end_date))

Michele Gatti
@mikeleg
May 15 2017 08:20
into a django drf is possible search by fields into a foreign key?
:]
@elcolie
May 15 2017 08:22
@badcoder28 I knew it. But it is cover all of my case
Roel
@roelzkie15
May 15 2017 08:25
@elcolie as i understand you want to get the booking dated from 21,22,23, and 24 by date range start_date=20 and end_date =25 right?
:]
@elcolie
May 15 2017 08:26
My FactoryBooking properties contain start_date and end_date as above
And it is possible to have paritial overlap and over overlap interval when I do filter
import datetime

import pytest
from model_mommy import mommy

from apps.factory_bookings.models import FactoryBooking


@pytest.fixture
def add_s1(db):
    return add_generic_factory_booking(1, 21)


@pytest.fixture
def add_s2(db):
    return add_generic_factory_booking(22, 23)


@pytest.fixture
def add_s3(db):
    return add_generic_factory_booking(24, 28)


@pytest.fixture
def add_s4(db):
    return add_generic_factory_booking(1, 29)


def add_generic_factory_booking(start: int, end: int):
    """
    Model Mommy make fixture in May 2017
    :param db: 
    :return: 
    """
    start_date = datetime.date(2017, 5, start)
    end_date = datetime.date(2017, 5, end)
    return mommy.make(FactoryBooking, _quantity=1, _fill_optional=True, start_date=start_date, end_date=end_date)
And this is WIP of my test. Please let me know If you know the way to shorten my task
def test_tail_overlap(add_s1):
    """ 
    :return: 
    """
    import pdb; pdb.set_trace()


def test_fully_inside(add_s2):
    pass

def test_head_overlap(add_s3):
    pass

def test_bigger_than_query(add_s4):
    pass
Roel
@roelzkie15
May 15 2017 08:50

yeah i get it

S1 = 1 - 21
S2 = 22-23
S3 = 25-28
S4 = 2-29

So that means you want to filter with date range 20-25 and it should get all the bookings right?

Roel
@roelzkie15
May 15 2017 08:59
why dont you try this concept (StartA <= EndB) and (EndA >= StartB) based on this answer
:]
@elcolie
May 15 2017 09:51
So that means you want to filter with date range 20-25 and it should get all the bookings right?
Yes. You are right
why dont you try this concept __(StartA <= EndB) and (EndA >= StartB)__ based on this [answer](http://stackoverflow.com/a/325964/6143656)
Very intersting. Thank you :D
ismail sarenkapic
@ismailnp96_twitter
May 15 2017 14:01
I successfully,manually sent an email to specific email address and activated the user but I cannot figure out how to:
1 Set submitted emails into recipient_list in my post_save_activation_receiver signal,since email field itself is defined in other model(if it is possible)?
2 Make this emails that I sent more secure, because gmail is treating them like potential fishing content.
from django.conf import settings
from django.contrib.auth.models import (
    BaseUserManager, AbstractBaseUser
)
from django.contrib import messages
from django.core.mail import send_mail
from django.conf import settings
from django.core.validators import RegexValidator
from django.db import models
from django.db.models.signals import post_save
from .utils import code_generator

USERNAME_REGEX = '^[a-zA-Z0-9.+-]*$'

class MyUserManager(BaseUserManager):
    def create_user(self, username, email, password=None):
        if not email:
            raise ValueError('Users must have an email address')

        user = self.model(
            username = username,
            email=self.normalize_email(email),
        )

        user.set_password(password)
        user.save(using=self._db)
        return user

    def create_superuser(self, username, email, password):
        user = self.create_user(
            username,
            email,
            password=password,
        )
        user.is_admin = True
        user.is_staff = True
        user.save(using=self._db)
        return user


class MyUser(AbstractBaseUser):
    username = models.CharField(
                max_length=255, 
                validators=[
                    RegexValidator(
                        regex = USERNAME_REGEX,
                        message = 'Username must be Alpahnumeric or contain any of the following: ". @ + -" ',
                        code='invalid_username'
                    )],
                unique=True,)
    email = models.EmailField(
        verbose_name='email address',
        max_length=255,
        unique=True, )
    zipcode   = models.CharField(max_length=120, default="92660")
    is_active = models.BooleanField(default=True)
    is_staff = models.BooleanField(default=False)
    is_admin = models.BooleanField(default=False)

    objects = MyUserManager()

    USERNAME_FIELD = 'username'
    REQUIRED_FIELDS = ['email']

    def get_full_name(self):
        # The user is identified by their email address
        return self.email

    def get_short_name(self):
        # The user is identified by their email address
        return self.email

    def __str__(self):              # __unicode__ on Python 2
        return self.email

    def has_perm(self, perm, obj=None):
        "Does the user have a specific permission?"
        # Simplest possible answer: Yes, always
        return True

    def has_module_perms(self, app_label):
        "Does the user have permissions to view the app `app_label`?"
        # Simplest possible answer: Yes, always
        return True

class ActivationProfile(models.Model):
    user    = models.ForeignKey(settings.AUTH_USER_MODEL)
    key     = models.CharField(max_length=120)
    expired = models.BooleanField(default=False)

    def save(self, *args, **kwargs):
        self.key = code_generator()
        super(ActivationProfile, self).save(*args, **kwargs)

def post_save_activation_receiver(sender, instance, created, *args, **kwargs):
    if created:
        #send email
        subject = 'Registration'
        message = "http://127.0.0.1:8000/activate/{0}".format(instance.key)
        from_email = settings.EMAIL_HOST_USER
        recipient_list = ['UserEmail']
        print(recipient_list)

        send_mail(subject, message, from_email, recipient_list,fail_silently=True)

post_save.connect(post_save_activation_receiver, sender=ActivationProfile)
Marlon Richert
@marlonrichert
May 15 2017 17:02
Hi! New to Django here. I have a nested for loop in a template, and this seems to cause an excessive amount of time on db requests. Django Dev Toolbar shows a lot of duplicate queries. How can I avoid this?
If I put it in terms of what I'm reading at https://docs.djangoproject.com/en/1.8/topics/db/aggregation/, I want to list all Publishers, show all attributes for each Publisher, and list all Book for each Publisher, and show all attributes for each Book. It's really slow when I loop that in the template. How can I make this faster?
John
@flyboy1565
May 15 2017 17:08
@marlonrichert , can you provide the code that is running the query
Marlon Richert
@marlonrichert
May 15 2017 17:17
@flyboy1565, one moment while I obfuscate it. #NDA
            {% for publisher in publisher_list.iterator %}
            <tr class="active">
                <td><input type="checkbox" name="select{{publisher.pk}}" id=""></td>
                <td>{{ publisher.name }}</td>
                <td>{{ publisher.authors.count }}</td>
                <td>{{ publisher.total_age_of_authors }}</td>
            </tr>
                {% for book in publisher.books.all %}
                <tr>
                    <td>&nbsp;</td>
                    <td>{{ book.title }}</td>
                    <td>{{ book.authors.count }}</td>
                    <td></td>
                </tr>
                {% endfor %}
            {% endfor %}
@flyboy1565
What seems to cause a lot of overhead are the calls to .count and the call to the .total_age_of_authors function, which does a sum in Python code.
Marlon Richert
@marlonrichert
May 15 2017 17:25
When I remove those, it runs about 20 times faster.
Ilya Borovitinov
@icehaunter
May 15 2017 17:25
Hey guys. Anyone knows how to filter queryset by item not being present in another queryset?
Basically queryset subtraction
@marlonrichert You could annotate your query
Marlon Richert
@marlonrichert
May 15 2017 17:30

@icehaunter, yeah, I just happened to notice that myself. :D

This seems to work: Publisher.objects.annotate(author_count=Count('authors'))

Thanks for rubber ducking. ;)
Marlon Richert
@marlonrichert
May 15 2017 18:05
Hm, actually, that's not working. :(
It compiles, but it also doesn't seem to produce any output in my template. 😕
In which file should I put the annotate call? @icehaunter?
Marlon Richert
@marlonrichert
May 15 2017 18:11
And what's the difference between annotate and aggregate?
Marlon Richert
@marlonrichert
May 15 2017 18:26
How do I get log messages visible in the Django Debug Toolbar?
Marlon Richert
@marlonrichert
May 15 2017 18:35

What could be the reason that this is not working?

Publisher.objects.all().annotate(book_count=Count('books'))

<td>{{ publisher.book_count }}</td>

Marlon Richert
@marlonrichert
May 15 2017 18:45

OK, I got the logging to work. I do this:

    pubs = Publisher.objects.all().annotate(Count('books'), books__sum=Sum('book__price'))

    logging.error(pubs[0].books__count)
    logging.error(pubs[0].books__sum)

and the logger clearly shows the right values. Yet when I do this in my template:

                <td>{{ pub.books__count }}</td>
                <td>{{ pub.books__sum }}</td>

I see only two empty table cells.

What could be the problem?

Marlon Richert
@marlonrichert
May 15 2017 19:01

Now I got it partially to work by putting it inside this method:

    def get_queryset(self):
        pubs = Publisher.objects.annotate(Count('books'), orders__sum=Sum('books__price'))
        Author.objects.annotate(Count('books'))
        return pubs
`

How do I get the annotated authors into the queryset?

Marlon Richert
@marlonrichert
May 15 2017 19:42
Using return list(chain(pubs, authors)) does not work. I just end up with no data on my page.
Marlon Richert
@marlonrichert
May 15 2017 19:55
pubs = Publisher.objects.annotate(Count('books'), orders__sum=Sum('books__price')).prefetch_related('authors', 'authors__books') seems to significantly speed up the call to .count :)
So, problem solved (for all practical purposes).