server: add health endpoint to metrics listener
There are already facilities to run the metrics listener on a fixed
port, so this makes for a useful endpoint to have metrics and a health
check in Kubernetes without exposing them to the world. In the long run,
I would like to remove the metrics endpoint from the public listener,
but that will take more work.
server: setup router dedicated to webhook processing
This introduces a new chi router to allow processing (in Go) webhook
calls coming from other SourceHut services.
It ensures that incoming messages are signed with the webhook private
key and if so, provides access both to the database and service
configuration.
Signed-off-by: Simon Martin <simon@nasilyan.com>
server: support multiple bind addresses
server: support setting build version and date information
This is a pre-requisite to the effort to make this information available
via the GraphQL API.
Co-authored-by: Conrad Hoffmann <ch@bitfehler.net>
Signed-off-by: Simon Martin <simon@nasilyan.com>
server: allow setting ports for metrics, pprof
With an eye towards Kubernetes, it is desirable to avoid having the API
pick random ports for anything. This commit introduces additional flags
for the API server to specify the metrics and pprof ports. This change
is backwards-compatible insofar as the default behavior is preserved if
the flags are not used.
The log output is changed to include the full address that the
respective listeners are listening on.
Note that using "0.0.0.0:1234" might not quite work as expected, because
Go [1]. But that does not change the default behavior, so I guess it's
fine.
[1] https://github.com/golang/go/issues/48723
Don't uppercase error messages
server: add error presenter to make context cancellation semantic
feature: add submodule + middleware for feature flags
Refactor config loading and server initialization
As is, LoadConfig() does some things that are not strictly related to
the configuration, such as parsing command line arguments. This has led
to a proliferation of different ways to load the config based on various
needs and also prevents tools that need a config but are not services to
use custom command line arguments.
This commit aims to decouple config loading from everything else and
do nothing but loading the config files.
On a high level, this commit:
- renames server.NewServer() to server.New()
- moves config.Debug and config.Addr into the server package
- moves crypto.InitCrypto() call into server.New()
- moves command line parsing into server.New(), using passed-in values
rather than os.Args
The only changes required for services would be changing
cfg := config.LoadConfig(":5100")
server := server.NewServer("meta.sr.ht", cfg)
to
cfg := config.LoadConfig()
server := server.New("meta.sr.ht", ":5100", cfg, os.Args)
All other tools will be switched to just LoadConfig() and, optionally, a
call to crypto.InitCrypto(). I managed to completely remove some global
state (addr) and at least make the rest private, so that users are
forced to use the designated functions.
The config module gained support for custom FS implementation, mainly
for testing.
server: enable GraphQL introspection
server: log GraphQL queries if running in debug mode
server: handle CORS in server
Currently CORS is handled in nginx in front of the API. That setup is
not very suitable for Kubernetes. Instead, handle CORS here, so we do
not need any intermediaries between the ingress and the APIs.
The CORS settings are taken from sr.ht-nginx/graphql.conf [1]
[1] https://git.sr.ht/~sircmpwn/sr.ht-nginx/tree/master/item/graphql.conf
server: disable logging for release mode
This was partially implemented a while ago but we missed the main point
where it matters.
server: add pprof endpoint
This adds a pprof endpoint to all SourceHut APIs. It picks an
OS-assigned port on localhost to run it on and prints this port to
stdout. This ensures a unique port is selected even if several APIs are
running on a single host.
server: enable request logging only in debug mode
To improve the signal:noise ratio in the production logs.
redis: avoid building UniversalOptions for standard mode
This makes it support all options that go-redis recognizes in ParseURL,
by constructing a standard client directly instead of building
UniversalOptions ourselves.
Export webhooks public key via api-meta.json
config: factorize integer parsing
Add a new GetInt() function to parse an integer value from a parsed
ini.File. Use that function instead of duplicated code.
Signed-off-by: Robin Jarry <robin@jarry.cc>
treewide: update to new multi worker dowork api
The work.Queue() implementation has changed and now allows scheduling
tasks from multiple goroutines in parallel. Also, it requires a new
argument for limiting the queue buffer size in order to apply back
pressure.
Define new configuration variables to allow tuning the queue sizes and
number of workers per service:
[mail]
# Maximum size of the outgoing email queue (default 512).
egress-queue-size = 512
[$service_name]
# Number of parallel workers per queue (default 1).
# There are multiple queues (for egress email, webhooks, etc.).
# This setting is applied on a per-queue basis.
queue-workers = 1
[webhooks]
# Maximum size of the webhooks queue (default 512).
queue-size = 512
Use these new settings to configure the queues and workers accordingly.
Fix unit tests as well.
Link: https://git.sr.ht/~sircmpwn/dowork/commit/95719cfc0118
Signed-off-by: Robin Jarry <robin@jarry.cc>