RE: need django.cache advice
Thanks, Javier.
The interesting 1000 records are changing very slowly. A few records could be added/deleted every minute but mostly it's some field changes to the records in the set.
No change is more or less worthwhile than any other change. All need to get into the database "eventually". Update is not expensive but regenerating the queryset is.
-----Original Message-----
From: django-users@googlegroups.com [mailto:django-users@googlegroups.com] On Behalf Of Javier Guerra Giraldez
Sent: Thursday, January 09, 2014 4:58 PM
To: django-users@googlegroups.com
Subject: Re: need django.cache advice
On Thu, Jan 9, 2014 at 4:43 PM, Sells, Fred <fred.sells@adventistcare.org> wrote:
>
> Fred: I was unclear. The initial join is 10000 x 1000 x 20 x 100 ... which gets reduced to about 1000 records. A few records are changed every second.
are the 'interesting' 1000 out-records always the same, or changing slowly? is it easy to test if any specifc in-record alters the result or not?
if yes on both, then you might invert the calculation, from a 'pull'
method (the current) to a 'push', where you keep the result precalculated and on each modification you test if it's worthwhile to redo it.
this assumes:
A.- the process is expensive to do every time for every client (that's the problem)
B.- it's cheap to verify if the new data alters the result (my initial
questions)
C.- the queries are frequent enough that few precalcs would be unseen.
(i guess so given the number of clients and rate of queries)
if condition C doesn't hold, but B does, then another thing to try:
- on each query, first check if the answer is already precalculated.
if it is, just get that
- if it's not, do the job and store in the cache
- on each input modification, check if it alters the result. if so, just delete the cache.
finally, if condition B doesn't hold, just store the result in a cache entry that expires in a minute. that means that the process would occur at most once a minute and the clients could get results that are at most a minute old.
--
Javier
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFkDaoQf0PDDAMTjCzUp92jHk%2BS34CDcMt9iJS4uRTCJf%2BLQKw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/DCD75728F460F14586086EA606E830820111A51276%40ACEVXCHMBX1001.ADVENTISTCORP.NET.
For more options, visit https://groups.google.com/groups/opt_out.
0 Comments:
Post a Comment
Subscribe to Post Comments [Atom]
<< Home