webhooks: implement internal webhook users
AUTH_INTERNAL requests previously could not register webhooks. This
commit adds the necessary changes to allow for this.
all: pass errors to panic, not strings
GraphQL's recovery middleware can't handle strings so it just logs a
very not useful <nil>
webhooks: add User-Agent to outgoing requests
client: add User-Agent to outgoing requests
email: fix partial bodies
Depending on the email and buffer size, cleartext.Close() and
body.Close() may happen *after* buf.Bytes() is called; producing
incomplete emails, interrupted in the middle of the body.
Make sure to close both *before* getting the buffer contents.
Fixes: db4d67a2a6fb ("email: fix empty emails sent on retries")
Signed-off-by: Robin Jarry <robin@jarry.cc>
email: fix malformed emails
go-message 0.18.2 contains a new implementation of the message line
wrapping algorithm. Without it, messages sometimes contain random line
breaks in the middle of the body.
Update to the latest tag.
Signed-off-by: Robin Jarry <robin@jarry.cc>
email: fix empty emails sent on retries
io.Reader objects can only be consumed once. On retry, re-reading from
them will return 0 bytes all the time.
Change NewTask to take in a bytes array instead of a reader. Create
a new reader on each retry.
Also remove SendRaw which has an inherent issue related to retry as
well. message.Entity is virtually a reader that can only be consumed
once.
Update EnqueueRaw to take a bytes array directly and have it call
NewTask() like EnqueueStd does.
Fixes: 78cfc5d87a7d ("email: allow sending raw emails")
Signed-off-by: Robin Jarry <robin@jarry.cc>
auth: use config.IsInternalIP
config: add global internal-ipnet setting
Add support for a general [sr.ht]internal-ipnet list of networks that
can be considered OK for internal operations.
The default is loopback addresses (127.0.0.0/8 and ::1/128) and private
routable unicast addresses (192.168.0.0/16, 10.0.0.0/8 and fc00::/7)
Since parsing IP networks and addresses can be costly, store the result
of [sr.ht]internal-ipnet into a global list of net.IPNet and
a convenience IsInternalIP() function to be called by services.
Signed-off-by: Robin Jarry <robin@jarry.cc>
server: add @admin directive implementation
This will be used at least by meta.sr.ht in the near future.
config: factorize integer parsing
Add a new GetInt() function to parse an integer value from a parsed
ini.File. Use that function instead of duplicated code.
Signed-off-by: Robin Jarry <robin@jarry.cc>
email: allow sending raw emails
The current email.Enqueue() function only allows sending messages with
the server's default settings.
In order to support sending any arbitrary message, we need to expose
a lower level API taking a raw message.Entity pointer. This new API will
be reused in a future reimplementation of the lists.sr.ht ingress agent
to forward messages to subscribers.
Add email.EnqueueRaw() that calls email.SendRaw(). Retry sending 10
times in the case of failure.
Signed-off-by: Robin Jarry <robin@jarry.cc>
email: keep smtp client connection alive
Instead of connecting and disconnecting each and every time we need to
send an email, keep the SMTP connection alive attached to a context
variable.
Before sending an email, check if the connection is still valid and
reconnect if it is not.
Signed-off-by: Robin Jarry <robin@jarry.cc>
treewide: update to new multi worker dowork api
The work.Queue() implementation has changed and now allows scheduling
tasks from multiple goroutines in parallel. Also, it requires a new
argument for limiting the queue buffer size in order to apply back
pressure.
Define new configuration variables to allow tuning the queue sizes and
number of workers per service:
[mail]
# Maximum size of the outgoing email queue (default 512).
egress-queue-size = 512
[$service_name]
# Number of parallel workers per queue (default 1).
# There are multiple queues (for egress email, webhooks, etc.).
# This setting is applied on a per-queue basis.
queue-workers = 1
[webhooks]
# Maximum size of the webhooks queue (default 512).
queue-size = 512
Use these new settings to configure the queues and workers accordingly.
Fix unit tests as well.
Link: https://git.sr.ht/~sircmpwn/dowork/commit/95719cfc0118
Signed-off-by: Robin Jarry <robin@jarry.cc>
webhooks: do not crash when expire is NULL
This is currently not possible to create a webhook without an expiry
date. But the SQL database schema does not forbid it. If in the future
we would like to allow webhooks to be created without an expiry date, we
shouldn't crash when fetching them.
Check for the validity of the Expiry field before checking its value. If
it is nil, assume that the webhooks has no expiry date.
Signed-off-by: Robin Jarry <robin@jarry.cc>
auth: reduce scope of user_type
auth: grant scoped access to anon internal auth
Internal auth is granted access to everything, whereas anonymous
internal auth is pretty restricted. This is mostly to avoid accidentally
hitting resolvers that require a logged-in user, however. Given that all
anon internal use cases are hard-coded and tested, this seems like a
pretty low risk. Allowing this will have the huge benefit of making much
more information available to anon internal queries, which will unlock
removing a bunch of awkward work-arounds we put in place.
Note, however, that this is also a work-around. It saves us from adding
yet more work-arounds to the GQL schema, and in the meantime a redesign
of the schema (especially the directives) is being worked on.