[GH-ISSUE #85] optimize session usage #84

New issue

Closed

opened 2026-02-26 09:35:30 +03:00 by kerem · 3 comments

kerem commented

2026-02-26 09:35:30 +03:00

Owner

Originally created by @ThomasWaldmann on GitHub (Nov 13, 2013).
Original GitHub issue: https://github.com/nsupdate-info/nsupdate.info/issues/85

we access the session rather often:

read accesses:

done all over the place (e.g in base.html, remove_stale_ips, log.py) - hard to avoid for main site

write accesses:

remove_stale_ips context processor writes every few minutes, triggers detection, which writes again.
for ipv4.BASEDOMAIN and ipv6.BASEDOMAIN, we run the full application (so it'll establish a separate session for these hostnames), but these hosts are only needed for IP detection and IP updates
session.save() calls - are they all necessary (do we need immediate saving? does it get saved at the end of the request anyway?) especially check HomeView's .save() call, it triggered "database is locked" rather regularly while I did some performance test with apachebench.

note about session stores:

db sessions: expensive
cookie sessions: can't work for the current code, we store the IPs we get via the fake img accesses into the session record (we get the sessionid via url) - but we can't(?) set the cookie there (not a domain cookie and domain cookies might be bad for security)
memcached: can be used for further tuning, but first optimize our code

Originally created by @ThomasWaldmann on GitHub (Nov 13, 2013). Original GitHub issue: https://github.com/nsupdate-info/nsupdate.info/issues/85 we access the session rather often: read accesses: - done all over the place (e.g in base.html, remove_stale_ips, log.py) - hard to avoid for main site write accesses: - remove_stale_ips context processor writes every few minutes, triggers detection, which writes again. - for ipv4.BASEDOMAIN and ipv6.BASEDOMAIN, we run the full application (so it'll establish a separate session for these hostnames), but these hosts are only needed for IP detection and IP updates - session.save() calls - are they all necessary (do we need immediate saving? does it get saved at the end of the request anyway?) especially check HomeView's .save() call, it triggered "database is locked" rather regularly while I did some performance test with apachebench. note about session stores: - db sessions: expensive - cookie sessions: can't work for the current code, we store the IPs we get via the fake img accesses into the session record (we get the sessionid via url) - but we can't(?) set the cookie there (not a domain cookie and domain cookies might be bad for security) - memcached: can be used for further tuning, but first optimize our code

kerem

2026-02-26 09:35:30 +03:00

closed this issue
added the
enhancement

scalability
labels

kerem commented

2026-02-26 09:35:31 +03:00

Author

Owner

@ThomasWaldmann commented on GitHub (Nov 14, 2013):

some optimizations / cleanups done by f6fb8b67bd

the "database is locked" seems to be an issue with concurrent accesses to sqlite database and seems to only happen under high load.

@ThomasWaldmann commented on GitHub (Nov 14, 2013): some optimizations / cleanups done by f6fb8b67bd1e5ddf26e0ba685c591ac4683e9450 the "database is locked" seems to be an issue with concurrent accesses to sqlite database and seems to only happen under high load.

kerem commented

2026-02-26 09:35:31 +03:00

Author

Owner

@ThomasWaldmann commented on GitHub (Nov 14, 2013):

it doesn't set a cookie / establish a session for ipv4/6.BASEDOMAIN/myip or /detectip - as we don't access request.session there. if one uses the full application on these hosts (other views), it does establish a session, of course.

@ThomasWaldmann commented on GitHub (Nov 14, 2013): it doesn't set a cookie / establish a session for ipv4/6.BASEDOMAIN/myip or /detectip - as we don't access request.session there. if one uses the full application on these hosts (other views), it does establish a session, of course.

kerem commented

2026-02-26 09:35:31 +03:00

Author

Owner

@ThomasWaldmann commented on GitHub (Nov 14, 2013):

the code part of this issue is done.

what remains is to use a more scalable DB and a more scalable session store on nsupdate.info site (postgreSQL?) and/or a better session store (memcached?). created #97 and #98 about that, so this issue can be closed.

@ThomasWaldmann commented on GitHub (Nov 14, 2013): the code part of this issue is done. what remains is to use a more scalable DB and a more scalable session store on nsupdate.info site (postgreSQL?) and/or a better session store (memcached?). created #97 and #98 about that, so this issue can be closed.