[GH-ISSUE #447] Nameserver Shared Secret shown is a repr() of a bytes object #324

Closed
opened 2026-02-26 10:30:50 +03:00 by kerem · 23 comments
Owner

Originally created by @abclution on GitHub (Dec 28, 2019).
Original GitHub issue: https://github.com/nsupdate-info/nsupdate.info/issues/447

Originally assigned to: @ThomasWaldmann on GitHub.

Finally managed to kludge along far enough to get nsupdate running on my local server. Have been using the nsupdate.info services for a few days so I know how it is supposed to work when working properly.

My local version, pulled from git is generating the shared secrets wrongly. Example:

Nameserver Shared Secret Generated
New nameserver shared secret generated for you. Everytime you visit this page a new secret will be generated and the old one becomes invalid.

Algorithm: hmac-md5

Secret: b'YXE2YXdjZVVGVlRkU0VyNQ=='

This is not a valid secret. A more valid secret is YXE2YXdjZVVGVlRkU0VyNQ==
Getting an extra b'
Not sure why its like that.

Oh and the most important part.
I am updating the same dns server (different domains) with my locally installed copy as I am with the nsupdate.info service. And it doesn't work.

request has invalid signature: TSIG

Probably cause of wrong key gen. I did try removing the b' but it still didnt work.

Update: Even after running the dnssec-keygen -a hmac-md5 -b 128 -n HOST my.dns.update.key. and setting up my server and my local nsupdate service, still doesn't work.

I'm going to try and revert to a stable release of nsupdate.info but git version as of today seems borked.

Originally created by @abclution on GitHub (Dec 28, 2019). Original GitHub issue: https://github.com/nsupdate-info/nsupdate.info/issues/447 Originally assigned to: @ThomasWaldmann on GitHub. Finally managed to kludge along far enough to get nsupdate running on my local server. Have been using the nsupdate.info services for a few days so I know how it is supposed to work when working properly. My local version, pulled from git is generating the shared secrets wrongly. Example: Nameserver Shared Secret Generated New nameserver shared secret generated for you. Everytime you visit this page a new secret will be generated and the old one becomes invalid. Algorithm: hmac-md5 Secret: b'YXE2YXdjZVVGVlRkU0VyNQ==' This is not a valid secret. A more valid secret is YXE2YXdjZVVGVlRkU0VyNQ== Getting an extra b' Not sure why its like that. Oh and the most important part. I am updating the same dns server (different domains) with my locally installed copy as I am with the nsupdate.info service. And it doesn't work. request has invalid signature: TSIG Probably cause of wrong key gen. I did try removing the b' but it still didnt work. Update: Even after running the dnssec-keygen -a hmac-md5 -b 128 -n HOST my.dns.update.key. and setting up my server and my local nsupdate service, still doesn't work. I'm going to try and revert to a stable release of nsupdate.info but git version as of today seems borked.
kerem closed this issue 2026-02-26 10:30:50 +03:00
Author
Owner

@ThomasWaldmann commented on GitHub (Dec 29, 2019):

likely a bug in the code when running on py3.

guess there is a .decode() missing.

On December 29, 2019 12:53:30 AM GMT+01:00, abclution notifications@github.com wrote:

Finally managed to kludge along far enough to get nsupdate running on
my local server. Have been using the nsupdate.info services for a few
days so I know how it is supposed to work when working properly.

My local version, pulled from git is generating the shared secrets
wrongly. Example:

`Nameserver Shared Secret Generated
New nameserver shared secret generated for you. Everytime you visit
this page a new secret will be generated and the old one becomes
invalid.

Algorithm: hmac-md5

Secret: b'YXE2YXdjZVVGVlRkU0VyNQ=='`

This is not a valid secret. A more valid secret is
YXE2YXdjZVVGVlRkU0VyNQ==

Not sure why its like that.

--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/nsupdate-info/nsupdate.info/issues/447

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

<!-- gh-comment-id:569496771 --> @ThomasWaldmann commented on GitHub (Dec 29, 2019): likely a bug in the code when running on py3. guess there is a .decode() missing. On December 29, 2019 12:53:30 AM GMT+01:00, abclution <notifications@github.com> wrote: >Finally managed to kludge along far enough to get nsupdate running on >my local server. Have been using the nsupdate.info services for a few >days so I know how it is supposed to work when working properly. > >My local version, pulled from git is generating the shared secrets >wrongly. Example: > >`Nameserver Shared Secret Generated >New nameserver shared secret generated for you. Everytime you visit >this page a new secret will be generated and the old one becomes >invalid. > >Algorithm: hmac-md5 > >Secret: b'YXE2YXdjZVVGVlRkU0VyNQ=='` > >This is not a valid secret. A more valid secret is >YXE2YXdjZVVGVlRkU0VyNQ== > >Not sure why its like that. > >-- >You are receiving this because you are subscribed to this thread. >Reply to this email directly or view it on GitHub: >https://github.com/nsupdate-info/nsupdate.info/issues/447 -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Author
Owner

@ThomasWaldmann commented on GitHub (Dec 29, 2019):

I can't reproduce this on nsupdate.info, there is no b'...', but just ....

<!-- gh-comment-id:569520560 --> @ThomasWaldmann commented on GitHub (Dec 29, 2019): I can't reproduce this on nsupdate.info, there is no `b'...'`, but just `...`.
Author
Owner

@ThomasWaldmann commented on GitHub (Dec 29, 2019):

What's your python version you use for running nsupdate.info?

<!-- gh-comment-id:569520600 --> @ThomasWaldmann commented on GitHub (Dec 29, 2019): What's your python version you use for running nsupdate.info?
Author
Owner

@abclution commented on GitHub (Dec 29, 2019):

Give me a couple mins to get you some more info.

<!-- gh-comment-id:569520618 --> @abclution commented on GitHub (Dec 29, 2019): Give me a couple mins to get you some more info.
Author
Owner

@abclution commented on GitHub (Dec 29, 2019):

Debian 9
Python 3.5.3
Running with Apache using WSGIDaemonProcess

I didnt check if Django inbuilt server does the same thing, but I can.

<!-- gh-comment-id:569520979 --> @abclution commented on GitHub (Dec 29, 2019): Debian 9 Python 3.5.3 Running with Apache using WSGIDaemonProcess I didnt check if Django inbuilt server does the same thing, but I can.
Author
Owner

@ThomasWaldmann commented on GitHub (Dec 29, 2019):

please check, i have same os / py.

On December 29, 2019 5:32:13 PM GMT+01:00, abclution notifications@github.com wrote:

Debian 9
Python 3.5.3
Running with Apache using WSGIDaemonProcess

I didnt check if Django inbuilt server does the same thing, but I can.

--
You are receiving this because you were assigned.
Reply to this email directly or view it on GitHub:
https://github.com/nsupdate-info/nsupdate.info/issues/447#issuecomment-569520979

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

<!-- gh-comment-id:569522196 --> @ThomasWaldmann commented on GitHub (Dec 29, 2019): please check, i have same os / py. On December 29, 2019 5:32:13 PM GMT+01:00, abclution <notifications@github.com> wrote: >Debian 9 >Python 3.5.3 >Running with Apache using WSGIDaemonProcess > >I didnt check if Django inbuilt server does the same thing, but I can. > > > >-- >You are receiving this because you were assigned. >Reply to this email directly or view it on GitHub: >https://github.com/nsupdate-info/nsupdate.info/issues/447#issuecomment-569520979 -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Author
Owner

@abclution commented on GitHub (Dec 29, 2019):

Same behavior using runserver.

Algorithm: hmac-md5
Secret: b'MlFwSDJwcUZSc1BBWDVDVQ=='

Algorithm: hmac-md5
Secret: b'MjhwSkg5ZlBITDJlZXZaZQ=='

etc.

<!-- gh-comment-id:569523428 --> @abclution commented on GitHub (Dec 29, 2019): Same behavior using runserver. Algorithm: hmac-md5 Secret: b'MlFwSDJwcUZSc1BBWDVDVQ==' Algorithm: hmac-md5 Secret: b'MjhwSkg5ZlBITDJlZXZaZQ==' etc.
Author
Owner

@ThomasWaldmann commented on GitHub (Dec 29, 2019):

url where you see that?

On December 29, 2019 6:02:18 PM GMT+01:00, abclution notifications@github.com wrote:

Same behavior using runserver.

Algorithm: hmac-md5
Secret: b'MlFwSDJwcUZSc1BBWDVDVQ=='

Algorithm: hmac-md5
Secret: b'MjhwSkg5ZlBITDJlZXZaZQ=='

etc.

--
You are receiving this because you were assigned.
Reply to this email directly or view it on GitHub:
https://github.com/nsupdate-info/nsupdate.info/issues/447#issuecomment-569523428

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

<!-- gh-comment-id:569523790 --> @ThomasWaldmann commented on GitHub (Dec 29, 2019): url where you see that? On December 29, 2019 6:02:18 PM GMT+01:00, abclution <notifications@github.com> wrote: >Same behavior using runserver. > >Algorithm: hmac-md5 >Secret: b'MlFwSDJwcUZSc1BBWDVDVQ==' > >Algorithm: hmac-md5 >Secret: b'MjhwSkg5ZlBITDJlZXZaZQ==' > >etc. > > > >-- >You are receiving this because you were assigned. >Reply to this email directly or view it on GitHub: >https://github.com/nsupdate-info/nsupdate.info/issues/447#issuecomment-569523428 -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Author
Owner

@abclution commented on GitHub (Dec 29, 2019):

http://localhost:8999/generate_ns_secret/1/?

<!-- gh-comment-id:569523838 --> @abclution commented on GitHub (Dec 29, 2019): http://localhost:8999/generate_ns_secret/1/?
Author
Owner

@abclution commented on GitHub (Dec 29, 2019):

I haven't pulled from git for ~24 hours i think my current commit/revision is

git rev-parse origin/master
ed673c68d0

<!-- gh-comment-id:569524342 --> @abclution commented on GitHub (Dec 29, 2019): I haven't pulled from git for ~24 hours i think my current commit/revision is git rev-parse origin/master ed673c68d02e81fa36495c56ecf84efe8780cd78
Author
Owner

@ThomasWaldmann commented on GitHub (Dec 29, 2019):

I still can't reproduce. Can you reproduce on https://nsupdate.info/?

<!-- gh-comment-id:569550510 --> @ThomasWaldmann commented on GitHub (Dec 29, 2019): I still can't reproduce. Can you reproduce on https://nsupdate.info/?
Author
Owner

@abclution commented on GitHub (Dec 30, 2019):

Hmm, don't think so. I tried again but on the nsupdate.info service, codes are generated correctly.
I will try using a "stable" release, and see if I get the same behavior.

But as noted before, it seems to be more than just the generation, even manually overriding the strings, the local version of nsupdate can not make a successful update with my dns server (request has invalid signature: TSIG), while the nsupdate.info service works to update domains (on the same dns server).

But will take some time I honestly went through the whole process of understanding the code and django all in one late night go, and was very painfully difficult, my deployment notes are the ramblings of a madman.

What libraries generate that code? Anything else I can check versions on? The only other thing of note is this machine is a LXC container, running on a Proxmox machine, and while the basic OS is Debian 9, I am also running Virtualmin on it to handle Apache virtualhosting etc.

My virtualenv was created with pip3 otherwise followed the instructions in the documentation, it also seems to have installed the Django 2.2.9 which I dont know how much of a difference that would be.

<!-- gh-comment-id:569641330 --> @abclution commented on GitHub (Dec 30, 2019): Hmm, don't think so. I tried again but on the nsupdate.info service, codes are generated correctly. I will try using a "stable" release, and see if I get the same behavior. But as noted before, it seems to be more than just the generation, even manually overriding the strings, the local version of nsupdate can not make a successful update with my dns server (request has invalid signature: TSIG), while the nsupdate.info service works to update domains (on the same dns server). But will take some time I honestly went through the whole process of understanding the code and django all in one late night go, and was very painfully difficult, my deployment notes are the ramblings of a madman. What libraries generate that code? Anything else I can check versions on? The only other thing of note is this machine is a LXC container, running on a Proxmox machine, and while the basic OS is Debian 9, I am also running Virtualmin on it to handle Apache virtualhosting etc. My virtualenv was created with pip3 otherwise followed the instructions in the documentation, it also seems to have installed the Django 2.2.9 which I dont know how much of a difference that would be.
Author
Owner

@abclution commented on GitHub (Dec 30, 2019):

Via pip freeze >

alabaster==0.7.12
apipkg==1.5
attrs==19.3.0
Babel==2.7.0
certifi==2019.11.28
chardet==3.0.4
defusedxml==0.6.0
Django==2.2.9
django-bootstrap-form==3.4
django-debug-toolbar==2.1
django-extensions==2.2.5
django-referrer-policy==1.0
django-registration-redux==2.6
dnspython==1.16.0
docutils==0.15.2
execnet==1.7.1
gunicorn==20.0.4
idna==2.8
imagesize==1.2.0
importlib-metadata==1.3.0
Jinja2==2.10.3
MarkupSafe==1.1.1
more-itertools==8.0.2
mysqlclient==1.4.6
netaddr==0.7.19
-e git+https://github.com/nsupdate-info/nsupdate.info.git@ed673c68d02e81fa36495c56ecf84efe8780cd78#egg=nsupdate
oauthlib==3.1.0
packaging==19.2
pathlib2==2.3.5
pep8==1.7.1
pkg-resources==0.0.0
pluggy==0.13.1
psycopg2==2.8.4
py==1.8.1
Pygments==2.5.2
PyJWT==1.7.1
pyparsing==2.4.6
pytest==5.3.2
pytest-cache==1.0
pytest-django==3.7.0
pytest-pep8==1.0.6
python3-openid==3.1.0
pytz==2019.3
requests==2.22.0
requests-oauthlib==1.3.0
setuptools-scm==3.3.3
six==1.13.0
snowballstemmer==2.0.0
social-auth-app-django==3.1.0
social-auth-core==3.2.0
Sphinx==2.3.1
sphinxcontrib-applehelp==1.0.1
sphinxcontrib-devhelp==1.0.1
sphinxcontrib-htmlhelp==1.0.2
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.2
sphinxcontrib-serializinghtml==1.1.3
sqlparse==0.3.0
urllib3==1.25.7
wcwidth==0.1.7
zipp==0.6.0

<!-- gh-comment-id:569651834 --> @abclution commented on GitHub (Dec 30, 2019): Via pip freeze > alabaster==0.7.12 apipkg==1.5 attrs==19.3.0 Babel==2.7.0 certifi==2019.11.28 chardet==3.0.4 defusedxml==0.6.0 Django==2.2.9 django-bootstrap-form==3.4 django-debug-toolbar==2.1 django-extensions==2.2.5 django-referrer-policy==1.0 django-registration-redux==2.6 dnspython==1.16.0 docutils==0.15.2 execnet==1.7.1 gunicorn==20.0.4 idna==2.8 imagesize==1.2.0 importlib-metadata==1.3.0 Jinja2==2.10.3 MarkupSafe==1.1.1 more-itertools==8.0.2 mysqlclient==1.4.6 netaddr==0.7.19 -e git+https://github.com/nsupdate-info/nsupdate.info.git@ed673c68d02e81fa36495c56ecf84efe8780cd78#egg=nsupdate oauthlib==3.1.0 packaging==19.2 pathlib2==2.3.5 pep8==1.7.1 pkg-resources==0.0.0 pluggy==0.13.1 psycopg2==2.8.4 py==1.8.1 Pygments==2.5.2 PyJWT==1.7.1 pyparsing==2.4.6 pytest==5.3.2 pytest-cache==1.0 pytest-django==3.7.0 pytest-pep8==1.0.6 python3-openid==3.1.0 pytz==2019.3 requests==2.22.0 requests-oauthlib==1.3.0 setuptools-scm==3.3.3 six==1.13.0 snowballstemmer==2.0.0 social-auth-app-django==3.1.0 social-auth-core==3.2.0 Sphinx==2.3.1 sphinxcontrib-applehelp==1.0.1 sphinxcontrib-devhelp==1.0.1 sphinxcontrib-htmlhelp==1.0.2 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.2 sphinxcontrib-serializinghtml==1.1.3 sqlparse==0.3.0 urllib3==1.25.7 wcwidth==0.1.7 zipp==0.6.0
Author
Owner

@ThomasWaldmann commented on GitHub (Dec 30, 2019):

The production site is currently on Django 1.11 (we are in the progress of updating it to 2.2.x and postgresql).

<!-- gh-comment-id:569734452 --> @ThomasWaldmann commented on GitHub (Dec 30, 2019): The production site is currently on Django 1.11 (we are in the progress of updating it to 2.2.x and postgresql).
Author
Owner

@abclution commented on GitHub (Dec 30, 2019):

Ok. I just used whichever Django pip gave me. So, duly noted. Will recreate my virtualenv and see what happens. :)

<!-- gh-comment-id:569749564 --> @abclution commented on GitHub (Dec 30, 2019): Ok. I just used whichever Django pip gave me. So, duly noted. Will recreate my virtualenv and see what happens. :)
Author
Owner

@abclution commented on GitHub (Dec 30, 2019):

Well, I guess I should ALSO mention, that everything else seems to work on the Django version I am using it with, EXCEPT for when I set it up with Apache, I could not get the default sql-lite db settings to work. Thought it was permissions or path problems, checked everything over and over, no luck.

OperationalError at /
unable to open database file, or a few other messages related to closed connections before completing the request. ( I forgot to write down that error)

Sqllite did work however with RunServer. Sooo.. weird stuff all around

In the end, as soon as I switched to Mysql/MariaDB it started working (where I found the other incompatibility.

<!-- gh-comment-id:569818090 --> @abclution commented on GitHub (Dec 30, 2019): Well, I guess I should ALSO mention, that everything else seems to work on the Django version I am using it with, EXCEPT for when I set it up with Apache, I could not get the default sql-lite db settings to work. Thought it was permissions or path problems, checked everything over and over, no luck. OperationalError at / unable to open database file, or a few other messages related to closed connections before completing the request. ( I forgot to write down that error) Sqllite did work however with RunServer. Sooo.. weird stuff all around In the end, as soon as I switched to Mysql/MariaDB it started working (where I found the other incompatibility.
Author
Owner

@abclution commented on GitHub (Dec 30, 2019):

The production site is currently on Django 1.11 (we are in the progress of updating it to 2.2.x and postgresql).

Just a question, why postgresql instead of mysql ? Or do you mean you are adding support for postgresql? (Fan of both, just asking your reasoning)

<!-- gh-comment-id:569819534 --> @abclution commented on GitHub (Dec 30, 2019): > The production site is currently on Django 1.11 (we are in the progress of updating it to 2.2.x and postgresql). Just a question, why postgresql instead of mysql ? Or do you mean you are adding support for postgresql? (Fan of both, just asking your reasoning)
Author
Owner

@lkraider commented on GitHub (Dec 31, 2019):

Adding that I could reproduce the issue with the built docker image as well (https://github.com/nsupdate-info/nsupdate.info/pull/441).

<!-- gh-comment-id:569855980 --> @lkraider commented on GitHub (Dec 31, 2019): Adding that I could reproduce the issue with the built docker image as well (https://github.com/nsupdate-info/nsupdate.info/pull/441).
Author
Owner

@abclution commented on GitHub (Jan 1, 2020):

Ok, its confirmed this bug is specifically due to Django 2.2.9 being used.
Switched to prior Django 1.11x and codes are generated correctly again.

BUUUUUT.... the plot thickens.

  1. There is no problem in the transmission of a secret using Django 2.2.9 as previously reported, there was a configuration error on my end so ignore that part of the issue.

  2. After switching to Django 1.x for testing, I switched back to Django 2.2.9 to see what other issues I may find while running Django 2.2.9 and .... SECRET generation is now working properly with Django 2.2.9

So.. I dunno WTF.

<!-- gh-comment-id:570081684 --> @abclution commented on GitHub (Jan 1, 2020): Ok, its confirmed this bug is specifically due to Django 2.2.9 being used. Switched to prior Django 1.11x and codes are generated correctly again. BUUUUUT.... the plot thickens. 1. There is no problem in the transmission of a secret using Django 2.2.9 as previously reported, there was a configuration error on my end so ignore that part of the issue. 2. After switching to Django 1.x for testing, I switched back to Django 2.2.9 to see what other issues I may find while running Django 2.2.9 and .... **SECRET generation is now working properly with Django 2.2.9** So.. I dunno WTF.
Author
Owner

@ThomasWaldmann commented on GitHub (Jan 1, 2020):

Huh? That sounds strange...

About postgres vs. mysql/mariadb: one can use anything that is supported as a django backend.

My impression is that postgresql is a superior db over mysql/mariadb and has stronger support from the python community, so I will use that.

<!-- gh-comment-id:570086483 --> @ThomasWaldmann commented on GitHub (Jan 1, 2020): Huh? That sounds strange... About postgres vs. mysql/mariadb: one can use anything that is supported as a django backend. My impression is that postgresql is a superior db over mysql/mariadb and has stronger support from the python community, so I will use that.
Author
Owner

@abclution commented on GitHub (Jan 1, 2020):

Yes, postgresql is quite good IMO.

Regarding the Django 1 to 2 to 1 and back to 2 shenanigans I don't know what to say, maybe @lkraider can see if they can get the same behavior. I used pip to uninstall/install the django versions from/into my previous virtualenv. Really weird and have not idea really how to explain the situation.

Happy New Years btw!

<!-- gh-comment-id:570087968 --> @abclution commented on GitHub (Jan 1, 2020): Yes, postgresql is quite good IMO. Regarding the Django 1 to 2 to 1 and back to 2 shenanigans I don't know what to say, maybe @lkraider can see if they can get the same behavior. I used pip to uninstall/install the django versions from/into my previous virtualenv. Really weird and have not idea really how to explain the situation. Happy New Years btw!
Author
Owner

@adrienb4 commented on GitHub (Jan 21, 2020):

Same problem as #453
With my colleague we have made a merge request to fix this.

Be carrefull with SQLite. You can easily exceed the maximum size of the field without error. In this case, domain secret field is theoretically limited to 88 (str mode). But her length in binary mode is 91.
That is why you are seeing b'XXXXXXXXXXXXXXXXXXX'

image

See: https://sqlite.org/faq.html#q9

With MySQL backend and domain secret in binary mode you have HTTP error because MySQL won't to store domain secret because her length is too long (max 88 vs 91).

image

<!-- gh-comment-id:576711294 --> @adrienb4 commented on GitHub (Jan 21, 2020): Same problem as #453 With my colleague we have made a merge request to fix this. Be carrefull with SQLite. You can easily exceed the maximum size of the field without error. In this case, domain secret field is theoretically limited to 88 (str mode). But her length in binary mode is 91. That is why you are seeing **b'XXXXXXXXXXXXXXXXXXX'** ![image](https://user-images.githubusercontent.com/15191572/72812763-68be8780-3c62-11ea-8f49-5bdd1387ce5d.png) See: https://sqlite.org/faq.html#q9 With MySQL backend and domain secret in binary mode you have HTTP error because MySQL won't to store domain secret because her length is too long (max 88 vs 91). ![image](https://user-images.githubusercontent.com/15191572/72813382-817b6d00-3c63-11ea-9bed-35fda57d0db9.png)
Author
Owner

@ThomasWaldmann commented on GitHub (Jan 23, 2020):

fixed by #454.

<!-- gh-comment-id:577447548 --> @ThomasWaldmann commented on GitHub (Jan 23, 2020): fixed by #454.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/nsupdate.info-nsupdate-info#324
No description provided.