mirror of
https://github.com/hoppscotch/hoppscotch.git
synced 2026-04-25 16:55:59 +03:00
[GH-ISSUE #3257] [bug]: Self hosted email send(for sending magic link) failed #1082
Labels
No labels
CodeDay
a11y
browser limited
bug
bug fix
cli
core
critical
design
desktop
discussion
docker
documentation
duplicate
enterprise
feature
feature
fosshack
future
good first issue
hacktoberfest
help wanted
i18n
invalid
major
minor
need information
need testing
not applicable to hoppscotch
not reproducible
pull-request
question
refactor
resolved
sandbox
self-host
spam
stale
testmu
wip
wont fix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/hoppscotch#1082
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @giripradip on GitHub (Aug 16, 2023).
Original GitHub issue: https://github.com/hoppscotch/hoppscotch/issues/3257
Is there an existing issue for this?
Current behavior
The continue with email and sending the magic links fails with error on both admin and in hoppscotch app.
I have setup the environment variable like following:
MAILER_SMTP_URL="smtps://noreply@mydomain.com:password@mail.mydomain.com"
MAILER_ADDRESS_FROM='"From mycompany Hoppscotch" noreply@mydomain.com'
I even tried with MailCatcher like explained in this tutorial.
MAILER_SMTP_URL="smtp://host.docker.internal:1025"
MAILER_ADDRESS_FROM='"From my company Hoppscotch" noreply@mydomain.com'
I can see the MailCatcher ui running on 1080 port.
But I got following issue on hoppscotch admin app:
https://hoppapi.mydomain.com/v1/auth/signin?origin=admin 500 (Internal Server Error)
Error: Request failed with status code 500
at tu (index.8d42fd3d.js:67:8560)
at nu (index.8d42fd3d.js:67:8743)
at XMLHttpRequest.m (index.8d42fd3d.js:68:16
{"statusCode":500,"message":"Internal server error"}
The docker container shows the following:
[Nest] 19 - 08/16/2023, 9:31:18 AM ERROR [ExceptionsHandler] email/failed
hoppscotch-backend | Error: email/failed
hoppscotch-backend | at throwErr (/usr/src/app/dist/utils.js:13:11)
hoppscotch-backend | at MailerService.sendEmail (/usr/src/app/dist/mailer/mailer.service.js:39:41)
hoppscotch-backend | at async AuthService.signInMagicLink (/usr/src/app/dist/auth/auth.service.js:153:9)
hoppscotch-backend | at async AuthController.signInMagicLink (/usr/src/app/dist/auth/auth.controller.js:36:31)
hoppscotch-backend | at async /usr/src/app/node_modules/.pnpm/@nestjs+core@9.2.1_@nestjs+common@9.2.1_@nestjs+platform-express@9.2.1_reflect-metadata@0.1.13_rxjs@7.6.0/node_modules/@nestjs/core/router/router-execution-context.js:46:28
hoppscotch-backend | at async /usr/src/app/node_modules/.pnpm/@nestjs+core@9.2.1_@nestjs+common@9.2.1_@nestjs+platform-express@9.2.1_reflect-metadata@0.1.13_rxjs@7.6.0/node_modules/@nestjs/core/router/router-proxy.js:9:17
I am using Alma Linux 9. Please let me know if I need to make any settings related to server or If I am missing something.
Steps to reproduce
For hoppscotch app
Environment
Production
Version
Self-hosted
@YAGrandGPBL commented on GitHub (Aug 16, 2023):
+1 me too have that trouble
@loban-v-navek commented on GitHub (Aug 17, 2023):
try MAILER_SMTP_URL="smtp://
@giripradip commented on GitHub (Aug 17, 2023):
I have tried but it is not working
@balub commented on GitHub (Aug 19, 2023):
Replace the existing value inside
MAILER_ADDRESS_FROMwith just your email like thisMAILER_ADDRESS_FROM=user@domain.com@selfing12 commented on GitHub (Aug 23, 2023):
I am also getting an error
@giripradip commented on GitHub (Aug 23, 2023):
@balub It didn't work. I got the same error as @selfing12
@balub commented on GitHub (Aug 23, 2023):
Can you share your .env file ?
@giripradip commented on GitHub (Aug 23, 2023):
Mailer config
MAILER_SMTP_URL="smtps://noreply@mydomain.com:password@mail.mydomain.com"
MAILER_ADDRESS_FROM=noreply@mydomain.com
@alne88 commented on GitHub (Aug 29, 2023):
I was having similar issue, was getting timeouts when connecting to smtp. Even though I specified port 587 in the URL it seemed to still default to port 465.
I was able to overcome this by updating NestMailerModule import
https://github.com/hoppscotch/hoppscotch/blob/main/packages/hoppscotch-backend/src/mailer/mailer.module.ts#L14
to:
@selfing12 commented on GitHub (Sep 6, 2023):
there are no timeouts, there is an error 500 and an error in the backend container
@Mvnrxck commented on GitHub (Sep 11, 2023):
Have the same problem, do you know how to solve this? I can't find anything about this Issue.
@giripradip commented on GitHub (Sep 12, 2023):
Not yet. It is still not working for me as well.
@a-camacho commented on GitHub (Sep 14, 2023):
Same issue here, any information ?
Don't really want to modify by hand my NestMailerModule. But if it's really needed ...
@philippos86 commented on GitHub (Sep 14, 2023):
Facing similar issue here ...
@mayzyo commented on GitHub (Sep 16, 2023):
Same issue here. Is this an issue only in the latest build? I don't mind reverting to an older build if that solves the problem. Can't even start using hoppscotch at the moment...
@mayzyo commented on GitHub (Sep 16, 2023):
My solution is to overwrite
/usr/src/app/packages/hoppscotch-backend/dist/mailer/mailer.module.jswith the script below. It effectively replaces the section usingMAIL_SMTP_URLenv variable with the env variables below, allowing for more flexibility in controlling the behaviour of nestjs mailer:SMTP_HOST=smtp.domain.comSMTP_PORT=465SMTP_IGNORE_TLS=falseSMTP_SECURE=trueSMTP_REJECT_UNAUTHORIZED=falseSMTP_USER=user@domain.comSMTP_PASS=passFor my issue, it was a self-signed certificate, which really only needed
tls: { rejectUnauthorized: false }but it was much easier to play around with the configuration and find the problem.It's pretty easy to replace a single file with docker, just add:
--volume /path/to/file/mailer.module.js:/usr/src/app/packages/hoppscotch-backend/dist/mailer/mailer.module.js@T-helper commented on GitHub (Sep 21, 2023):
Thanks @mayzyo, your solution is working for me with gmail smtp authorization well ! It would be good to see in main branch
@a-camacho commented on GitHub (Sep 21, 2023):
Thanks, it works for me (for solving SMTP problem).
But deploying again a new AIO stack, I got the following error :
The tablepublic.Userdoes not exist in the current database.Could fix it using prisma migration command :
pnpx prisma migrate deployWeird I didn't get it last deploys.
@martindemker commented on GitHub (Sep 26, 2023):
Thanks @mayzyo, it worked perfectly. Hopefully this gets ported into the main branch.
@gilPts commented on GitHub (Oct 9, 2023):
Got the same issue, since i am using docker AIO i'm blocked.
Regards,
@alastair-drong-wd commented on GitHub (Oct 13, 2023):
Also facing this issue. I kinda' wish there was a generic OAuth option I could pick so I could point my instance at my existing Dex identity provider.
@bcoquell commented on GitHub (Oct 17, 2023):
Hello, using the AIO image, I was also facing issue in my case I have a simple STMP server without authentication needs, but no access on the server so I can't see the logs on server side and the generic error around email failure (Error: email/failed) was not too detailed to understand. So I follow the steps :
In the end, when I managed to add logs, I was able to see that the SMTP server was indeed contacted but it systematically received credentials which interrupted the SMTP exchange (simple SMTP with no authentication in my case).
I fix this issue by commenting the lines in the mailer.module.js (sure it has to be done with a best way sine it will break the authentication when needed) :
and it works now :)
Finally I replace the mailer.module.js with this version (based on @mayzyo one), which put auth json block only when the SMTP_SECURE env variable is set to true :
hope it helps other people
BR,
Baptiste
@WestFarmer commented on GitHub (Oct 24, 2023):
same here
@nmella commented on GitHub (Oct 24, 2023):
+1
@atefhaloui commented on GitHub (Oct 27, 2023):
Won't work even with the patch on my postfix instance :( got Timeout followed by a 504.
Looking at postfix logs, there is no connection established to the server even if a telnet is working perfectly.
My msmtp configuration:
@jobartim44 commented on GitHub (Nov 3, 2023):
I'am facing the same problem, logs are not explicits.
When using a local mail service without authentication, it's work but with a real SMTP server with authentication it always fails. I tried with an internal Exchange server, with Mailjet, and with Gmail.
@jobartim44 commented on GitHub (Nov 6, 2023):
Hi @liyasthomas I can't create a pull request to integrate @mayzyo code that works for every people wo tested it. Is it possible to do it ?
Thanks
@v01t commented on GitHub (Nov 12, 2023):
see here you may need to set
secure: falseandrequireTLS: truelike in my case,also adding
console.log(error.toString());into catch of error in mailer.service.js cold give more hints (response to client from server)@muellerst-hg commented on GitHub (Nov 14, 2023):
Pitfalls:
When using
MAILER_SMTP_URLfor transport configuration, keep in mind to escape/urlencode any value, especially usernames and passwords. Otherwise the underlying nodemailer might parse the URL incorrectly, using false or incomplete transport options, e.g. missing host or port options. It will definitely fail, if your password contains/, probably also with other reserved URL characters like:or@Some mail servers reject mails or flag them as spam, when
MAILER_ADDRESS_FROMdoes not follow the patternNAME <ADDRESS@FQDN>Debugging:
MAILER_SMTP_URLto enable logging and to debug the SMTP conversation:?logging=true&debug=true@bforbis commented on GitHub (Nov 14, 2023):
So I just found out that you can in fact configure the
tls: { rejectUnauthorized: false }configuration via the connection string, it's just that thenodemailerdocumentation does not document this so I don't blame anyone who didn't find this.You can look at the connection string parsing code here
TL;DR: Use the following to configure rejectUnauthorized
@alastair-drong-wd commented on GitHub (Nov 15, 2023):
Unfortunately, this formatting did not work for me 😑
@bforbis commented on GitHub (Nov 15, 2023):
Try also adding
logger=true&debug=true. This will cause nodemailer to print out more info.You can also test out
nodemaileryourself in a sample / throwaway npm project if that's easier than doing this through hoppscotch docker. That's how I figured all this out myself@muellerst-hg commented on GitHub (Nov 15, 2023):
Correct params are
logging=true&debug=trueRegarding nodemailer testing: Instead of sending mails with
sendMail()for testing, you could just verify the connection (incl. auth) with:@WestFarmer commented on GitHub (Nov 21, 2023):
I am using valid let's encrypt certs, still got this error.
Can someone explain why ?
my env:
@v01t commented on GitHub (Nov 21, 2023):
Hello,
I will extend my previous comment to give some more details
After digging into code and all comments above (thanks for those), I end up with following changes that works in my SMTP configuration case (STARTTLS):
/usr/src/app/packages/hoppscotch-backend/dist/mailer/mailer.module.js:
/usr/src/app/packages/hoppscotch-backend/dist/mailer/mailer.service.js:
I add
console.log(error.toString());just for debug purposes, as that's is where real error response from mail server could be seenenvs, just because they are a bit tricky for STARTTLS:
hope that with above info someone will save hours without digging into problem like it was for me ;)
so above could be example of required changes to support STARTTLS
Regards
@WestFarmer commented on GitHub (Nov 22, 2023):
after append options you guys mentioned like this:
MAILER_SMTP_URL: "smtps://hoppscotch@itps.xxx.com:******@smtp.itps.xxx.com:587?logger=true&debug=true&secure=false&requireTLS=true"I am seeing this:
@muellerst-hg commented on GitHub (Nov 22, 2023):
If your smtp server relies on STARTTLS instead of TLS/SSL , use
smtp://. Port 587 usually isused for this scenario.If your smtp server requires TLS/SSL , use
smtps://Port 465 usually isused for this scenarioHere are some short explanations: https://mailtrap.io/blog/starttls-ssl-tls/
This error indicates that your server does not support SSL/TLS on port 587. Try using
smtp://Btw. you better rotate secrets, Github sends email notifications as soon as a comment was added.
@simpleway commented on GitHub (Jan 15, 2024):
Have the same issue:
Backend Server | [Nest] 57 - 01/15/2024, 5:28:40 AM ERROR [ExceptionsHandler] email/failed
Backend Server | Error: email/failed
Backend Server | at throwErr (/usr/src/app/packages/hoppscotch-backend/dist/utils.js:14:11)
Backend Server | at MailerService.sendEmail (/usr/src/app/packages/hoppscotch-backend/dist/mailer/mailer.service.js:39:41)
Backend Server | at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Backend Server | at async AuthService.signInMagicLink (/usr/src/app/packages/hoppscotch-backend/dist/auth/auth.service.js:159:9)
Backend Server | at async AuthController.signInMagicLink (/usr/src/app/packages/hoppscotch-backend/dist/auth/auth.controller.js:46:31)
Backend Server | at async /usr/src/app/node_modules/.pnpm/@nestjs+core@10.2.7_@nestjs+common@10.2.7_@nestjs+platform-express@10.2.7_reflect-metadata@0.1.13_rxjs@7.6.0/node_modules/@nestjs/core/router/router-execution-context.js:46:28
Backend Server | at async /usr/src/app/node_modules/.pnpm/@nestjs+core@10.2.7_@nestjs+common@10.2.7_@nestjs+platform-express@10.2.7_reflect-metadata@0.1.13_rxjs@7.6.0/node_modules/@nestjs/core/router/router-proxy.js:9:17
When I add
MAILER_SMTP_URL=smtp://smtp.xxxxx.com:25?logger=true&debug=true(no authentication), I don't see any additional logs print out. I'm running AIO@2023.12.2 on kubernetes. Have I missed something?@mayzyo commented on GitHub (Jan 15, 2024):
Just an update to the file in my previously posted method. I stumbled upon my container updated to 2023.12.1 breaking the file.
Here's the version of mailer.module.js that would work with this build of Hoppscotch. Everything else stays the same as my previous post:
@Shaddowrunner414 commented on GitHub (Jan 24, 2024):
Hi there, I'm trying to setup hoppscotch community and also ran into a problem with the first signup with the magic link email.
No matter if I try it in the Admin Dashboard or the standard frontend, I get the error "request failed with status code 500".
I'm using the AIO Container, but I also tried the individual ones with the same result.
Our hMail Server does not recieve any request from the Hoppscotch-VM so the error is with hoppscotch or more likely the way I configured it. (It does not need any credentials.)
I also tried to use this fix (https://github.com/hoppscotch/hoppscotch/issues/3257#issuecomment-1891609112) with no improvements.
@mayzyo could you maybe let me know, how your variables should be configured if no authentication / credentials are needed? (I tried leaving them emtpy, commenting them, etc.)
I added the logging bits from (https://github.com/hoppscotch/hoppscotch/issues/3257#issuecomment-1766629473) and the error is a "Connection Timeout".
Using the IP of the Mailserver does not make a difference and nslookup works in the VM and the docker container.
I also set up a simple NodeJS Mail Service which worked fine and I also could integrate it into hoppscotch.
Then I got the mail delivery working but no content (like the magic link) was sent with these emails.
(Of course I also tried the exact same configuration that worked for the standalone mailservice in hoppscotch)
Could you help me out with the configuration?
@mayzyo commented on GitHub (Jan 28, 2024):
@Shaddowrunner414 Sure thing. So my own configuration is as follows:
I have a email server that enforces TLS but the SSL certificate is self generated. In my case, I mainly needed the IGNORE_TLS. If you say you have no security (check your provider docs) I would assume you need to change the port to one of the insecure ports and set SECURE to false maybe. I'd leave a '' for the password, but I haven't had any experience with what you are using so can't say for sure.
Hope that helps.
@Dimutrulkin commented on GitHub (Jan 30, 2024):
Hi.
We are trying to start using the service, but we are stuck with the same error. We specify the following mail string setting:
but we get an error
There are no Debug logs or error text are displayed.
Is it possible to perform the initial setup of the service without MagicLink and EMail? Is it possible to create the first administrator users in the database manually?
@LucHeart commented on GitHub (Jan 30, 2024):
I tried everything from this thread. Nothing works. Can we please just have a option to ignore SSL checks?
@LucHeart commented on GitHub (Jan 30, 2024):
Update: After looking around in the database a bit, it seems to save the settings there aswell, and does not update them from the env vars. So i dropped the database and recreated it... After that, everything works now.
/?secure=false&tls.rejectUnauthorized=false&logger=true&debug=trueas arguments after the url tho. Dont know if important to my issue at all, but it sure as hell works.@dmitrycrivoy commented on GitHub (Feb 26, 2024):
Thanks a lot, worked for me
@banfushen commented on GitHub (Mar 29, 2024):
I had the same problem, i solved it ok this way
.env file is the same as before
@devops-fleks commented on GitHub (Apr 25, 2024):
@banfushen is there any option that you can share the .env with the url composition? are you using mailcatcher?
@ztom99 commented on GitHub (Jun 12, 2024):
I also have this problem,and I start the program in my local vscode, I finally solve the problem by 2 steps :
1、modify the email setting in .env
Mailer config
MAILER_SMTP_URL=smtps://myEmailName:myAuthCode@smtp.domain.com # myEmailName doesn't need to have @domain.com, it's just the email name
2、drop the database,rebuild the database hoppscotch and migrate these tables
@aryaag commented on GitHub (Sep 13, 2024):
I was getting a few errors related to email and SMTP.
Both of these were fixed by removing SSL restrictions. Sharing my
.envconfig below:See
MAILER_SMTP_SECURE&MAILER_TLS_REJECT_UNAUTHORIZEDabove.@mrambossek commented on GitHub (Sep 14, 2024):
getting the same error as last post, with
hoppscotch/hoppscotch-backend latest 3486100de10c 2 days ago 486MB
with these settings
what is even weirder - it does not even connect at all to my host - tcpdump on the relay confirms that no connection is being made. but the settings are not, at least not completely, being ignored; if i set CUSTOM_CONFIGS to false, i get Error: auth/provider_not_configured_correctly .. so it definitely uses them.
@Lifeni commented on GitHub (Sep 23, 2024):
I also encountered the problem of
wrong version number. After checking the code:github.com/hoppscotch/hoppscotch@e9e1366cc8/packages/hoppscotch-backend/src/mailer/helper.ts (L36-L56)I found that this was because the modified environment variable value was not used, and the program still had the old environment variable. (After the
.envfile is modified, just recreate hoppscotch, but the environment variables underenv.INFRAare not changed)Therefore, to solve this problem, you need to run
docker volume rm <hoppscotch-db>, then rundocker compose up -dandpnpx prisma migrate deployto redeploy hoppscotch..env:docker-compose.yml@mrambossek commented on GitHub (Sep 23, 2024):
d'oh. thanks for this. i am not using the AIO image, but yeah wiping the database and THEN restarting with the correct .env fixed it for me as well...
@iDschepe commented on GitHub (Nov 27, 2024):
Important:
To apply the configuration from the .env file or Docker Compose to the database, the database must be deleted first.
Alternatively, you can manually update the changes directly in the database. After that, the Docker container needs to be restarted.
Important:
After the first admin login attempt, the user will appear in the database. You can then set the "isAdmin" column to "true" for that user to make them an admin.