Commit graph

4091 commits

Author SHA1 Message Date
Lunny Xiao
745b45406d
Fix context cache bug & enable context cache for dashabord commits' authors(#26991) (#27017)
backport #26991

Unfortunately, when a system setting hasn't been stored in the database,
it cannot be cached.
Meanwhile, this PR also uses context cache for push email avatar display
which should avoid to read user table via email address again and again.

According to my local test, this should reduce dashboard elapsed time
from 150ms -> 80ms .

(cherry picked from commit 9df573bddc)
2023-09-20 12:50:46 +02:00
wxiaoguang
8f6d442a04
Use secure cookie for HTTPS sites (#26999) (#27013)
Backport #26999

If the AppURL(ROOT_URL) is an HTTPS URL, then the COOKIE_SECURE's
default value should be true.

And, if a user visits an "http" site with "https" AppURL, they won't be
able to login, and they should have been warned. The only problem is
that the "language" can't be set either in such case, while I think it
is not a serious problem, and it could be fixed easily if needed.

(cherry picked from commit b0a405c5fa)
2023-09-20 12:50:46 +02:00
Giteabot
957a64d91a
Fix INI parsing for value with trailing slash (#26995) (#27001)
Backport #26995 by @wxiaoguang

Fix #26977 (a temp fix)

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit da7d7e60d8)
2023-09-20 12:50:46 +02:00
Gusted
4b9a473e12
[GITEA] Use restricted sanitizer for repository description
- Backport of https://codeberg.org/forgejo/forgejo/pulls/1433
  - Currently the repository description uses the same sanitizer as a
normal markdown document. This means that element such as heading and
images are allowed and can be abused.
  - Create a minimal restricted sanitizer for the repository description,
which only allows what the postprocessor currently allows, which are
links and emojis.
  - Added unit testing.
  - Resolves https://codeberg.org/forgejo/forgejo/issues/1202
  - Resolves https://codeberg.org/Codeberg/Community/issues/1122

(cherry picked from commit a8afa4cd18)
2023-09-13 17:17:37 +02:00
wxiaoguang
9c0380fe84
Avoid double-unescaping of form value (#26853) (#26863)
Backport #26853

The old `prepareQueryArg` did double-unescaping of form value.

(cherry picked from commit e8da63c24e)
2023-09-08 08:09:18 +02:00
Giteabot
d5845521a8
check blocklist for emails when adding them to account (#26812) (#26831)
Backport #26812 by @techknowlogick

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
(cherry picked from commit 41bae29f84)
2023-09-08 08:09:18 +02:00
Giteabot
0b1175f21b
Add fix incorrect can_create_org_repo for org owner team (#26683) (#26791)
Backport #26683 by @yp05327

Related to: #8312 #26491

In migration v109, we only added a new column `CanCreateOrgRepo` in Team
table, but not initial the value of it.
This may cause bug like #26491.

Co-authored-by: yp05327 <576951401@qq.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit c3d323fd85)
2023-09-08 08:09:18 +02:00
Giteabot
4d2b4008d3
Fix some slice append usages (#26778) (#26798)
Backport #26778 by @harryzcy

Co-authored-by: Chongyi Zheng <git@zcy.dev>
Co-authored-by: delvh <dev.lh@web.de>
(cherry picked from commit 4013f3f600)
2023-09-08 08:09:18 +02:00
Giteabot
06c45d3b6e
Use correct minio error (#26634) (#26639)
Backport #26634 by @delvh

Previously, `err` was defined above, checked for `err == nil` and used
nowhere else.
Hence, the result of `convertMinioErr` would always be `nil`.
This leads to a NPE further down the line.
That is not intentional, it should convert the error of the most recent
operation, not one of its predecessors.

Found through
https://discord.com/channels/322538954119184384/322538954119184384/1143185780206993550.

Co-authored-by: delvh <dev.lh@web.de>
(cherry picked from commit a4b14638b5)
2023-09-08 08:07:19 +02:00
Giteabot
19a49e763a
Add branch_filter to hooks API endpoints (#26599) (#26632)
Backport #26599 by @yardenshoham

We now include the branch filler in the response.

- Closes #26591

Signed-off-by: Yarden Shoham <git@yardenshoham.com>
Co-authored-by: Yarden Shoham <git@yardenshoham.com>
(cherry picked from commit fe78aabc67)
2023-09-08 08:07:19 +02:00
Earl Warren
d3b8870700
[TESTS] verify facts for the admin storage documentation (squash)
(cherry picked from commit d83d8ce57b)
2023-08-31 15:32:22 +02:00
Gusted
fa25b9eec6
[GITEA] Add slow SQL query warning
- Backport of https://codeberg.org/forgejo/forgejo/pulls/1284
  - Databases are one of the most important parts of Forgejo, every
interaction with Forgejo uses the database in one way or another.
Therefore, it is important to maintain the database and recognize when
Forgejo is not doing well with the database. Forgejo already has the
option to log *every* SQL query along with its execution time, but
monitoring becomes impractical for larger instances and takes up
unnecessary storage in the logs.
  - Add a QoL enhancement that allows instance administrators to specify a
threshold value beyond which query execution time is logged as a warning
in the xorm logger. The default value is a conservative five seconds to
avoid this becoming a source of spam in the logs.
  - The use case for this patch is that with an instance the size of Codeberg, monitoring SQL logs is not very fruitful and most of them are uninteresting. Recently, in the context of persistent deadlock issues (https://codeberg.org/forgejo/forgejo/issues/220), I have noticed that certain queries hold locks on tables like comment and issue for several seconds. This patch helps to identify which queries these are and when they happen.
  - Added unit test.
2023-08-21 21:18:43 +02:00
Giteabot
4ac522c8aa
Sync repo's IsEmpty status correctly (#26517) (#26560)
Backport #26517 by @wxiaoguang

Close #26509

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 7da85fa0c3)
2023-08-21 07:27:20 +02:00
Giteabot
622ec5c79f
Detect ogg mime-type as audio or video (#26494) (#26505)
Backport #26494 by @wxiaoguang

"ogg" is just a "container" format for audio and video.

Golang's `DetectContentType` only reports "application/ogg" for
potential ogg files.

Actually it could do more "guess" to see whether it is a audio file or a
video file.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 4bdb8dd9cc)
2023-08-21 07:26:43 +02:00
Giteabot
b683b93d16
Fix storage path logic especially for relative paths (#26441) (#26481)
Backport #26441 by @lunny

This PR rewrites the function `getStorage` and make it more clear.

Include tests from #26435, thanks @earl-warren

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Earl Warren <contact@earl-warren.org>
(cherry picked from commit f1c5d33d3e)
2023-08-21 07:22:19 +02:00
Giteabot
f74522a352
Close stdout correctly for "git blame" (#26470) (#26473)
Backport #26470 by @wxiaoguang

Close stdout correctly for "git blame", otherwise the failed "git blame"
would cause the request hanging forever.

And "os.Stderr" should never (seldom) be used as git command's stderr
(there seems some similar problems in code, they could be fixed later).

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit fe1b11b639)
2023-08-21 07:22:19 +02:00
Giteabot
0c8a96896f
Remove last newline from config file (#26468) (#26471)
Backport #26468 by @wxiaoguang

When users put the secrets into a file (GITEA__sec__KEY__FILE), the
newline sometimes is different to avoid (eg: echo/vim/...)

So the last newline could be removed when reading, it makes the users
easier to maintain the secret files.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 80d7288ea4)
2023-08-21 07:22:19 +02:00
Giteabot
82e5247a43
Check first if minio bucket exists before trying to create it (#26420) (#26465)
Backport #26420 by @lunny

For some reason, the permission of the client_id and secret may cannot
create bucket, so now we will check whether bucket does exist first and
then try to create a bucket if it doesn't exist.

Try to fix #25984

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
(cherry picked from commit 2d1202b32c)
2023-08-21 07:22:19 +02:00
Earl Warren
1b568e284f
[GITEA] add GetFile to config provider
(cherry picked from commit 88d1b53eea)
2023-08-21 07:22:19 +02:00
Earl Warren
03f33a0320
[SEMVER] store SemVer in ForgejoSemVer after a database upgrade
(cherry picked from commit b7fe7cf401)
2023-08-21 07:22:18 +02:00
Giteabot
3d69647e06
minio: add missing region on client initialization (#26412) (#26438)
Backport #26412 by @nekrondev

The MinIO client isn't redirecting to the correct AWS endpoint if a
non-default data center is used.

In my use case I created an AWS bucket at `eu-central-1` region. Because
of the missing region initialization of the client the default
`us-east-1` API endpoint is used returning a `301 Moved Permanently`
response that's not handled properly by MinIO client. This in return
aborts using S3 storage on AWS as the `BucketExists()` call will fail
with the http moved error.

MinIO client trace shows the issue:

```text
---------START-HTTP---------
HEAD / HTTP/1.1
Host: xxxxxxxxxxx-prod-gitea-data.s3.dualstack.us-east-1.amazonaws.com
User-Agent: MinIO (windows; amd64) minio-go/v7.0.61
Authorization: AWS4-HMAC-SHA256 Credential=**REDACTED**/20230809/accesspoint.eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230809T141143Z

HTTP/1.1 301 Moved Permanently
Connection: close
Content-Type: application/xml
Date: Wed, 09 Aug 2023 14:11:43 GMT
Server: AmazonS3
X-Amz-Bucket-Region: eu-central-1
X-Amz-Id-2: UK7wfeYi0HcTcytNvQ3wTAZ5ZP1mOSMnvRZ9Fz4xXzeNsS47NB/KfFx2unFxo3L7XckHpMNPPVo=
X-Amz-Request-Id: S1V2MJV8SZ11GEVN
---------END-HTTP---------
```

Co-authored-by: nekrondev <heiko@noordsee.de>
Co-authored-by: Heiko Besemann <heiko.besemann@qbeyond.de>
(cherry picked from commit 981ab48503)
2023-08-21 07:22:18 +02:00
wxiaoguang
e64aa0d9c2
Fix admin queue page title and fix CI failures (#26409) (#26421)
Backport #26409

* Fix #26408
* Bypass the data race issue in "ssh" package

(cherry picked from commit 8ad331c9d2)
2023-08-21 07:22:18 +02:00
Giteabot
8265bece8e
Introduce ctx.PathParamRaw to avoid incorrect unescaping (#26392) (#26405)
Backport #26392 by @wxiaoguang

Fix #26389

And complete an old TODO: `ctx.Params does un-escaping,..., which is
incorrect.`

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 2d1a7e1cd4)
2023-08-21 07:22:18 +02:00
Loïc Dachary
916ec9acab
[TESTS] MockVariable temporarily replaces a global value
defer test.MockVariable(&variable, 1234)()

(cherry picked from commit 9c78752444)
(cherry picked from commit 8ab559df0d)
(cherry picked from commit 2e7fe1ec95)
(cherry picked from commit f9618b8896)

Conflicts:
	modules/test/utils.go
	https://codeberg.org/forgejo/forgejo/issues/1219
2023-08-21 07:22:17 +02:00
wxiaoguang
2e539d5190
Fix incorrect CLI exit code and duplicate error message (#26346) (#26347)
Backport #26346

Follow the CLI refactoring, and add tests.

(cherry picked from commit fa431b377d)
2023-08-21 07:22:17 +02:00
Earl Warren
149cd865ea
Revert "[TESTS] MockVariable temporarily replaces a global value"
This reverts commit f9618b8896.
2023-08-21 07:22:17 +02:00
Giteabot
75417ed070
Prevent newline errors with Debian packages (#26332) (#26342)
Backport #26332 by @KN4CK3R

Fixes #26313

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit 3e9475b3b2)
2023-08-21 07:22:16 +02:00
Giteabot
e5c26e38f5
Make git batch operations use parent context timeout instead of default timeout (#26325) (#26330)
Backport #26325 by @wxiaoguang

Fix #26064

Some git commands should use parent context, otherwise it would exit too
early (by the default timeout, 10m), and the "cmd.Wait" waits till the
pipes are closed.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 9451781ebe)
2023-08-21 07:22:16 +02:00
Giteabot
5aa6a8288d
Fix the wrong derive path (#26271) (#26318)
Backport #26271 by @lunny

This PR will fix #26264, caused by #23911.

The package configuration derive is totally wrong when storage type is
local in that PR.

This PR fixed the inherit logic when storage type is local with some
unit tests.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 88f6f7579c)
2023-08-21 07:22:16 +02:00
Giteabot
db326835e6
Support getting changed files when commit ID is EmptySHA (#26290) (#26316)
Backport #26290 by @Zettat123

Fixes #26270.

Co-Author: @wxiaoguang

Thanks @lunny for providing this solution

As
https://github.com/go-gitea/gitea/issues/26270#issuecomment-1661695151
said, at present we cannot get the names of changed files correctly when
the `OldCommitID` is `EmptySHA`. In this PR, the `GetCommitFilesChanged`
method is added and will be used to get the changed files by commit ID.

References:
- https://stackoverflow.com/a/424142

Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit a57568bad7)
2023-08-21 07:22:16 +02:00
Giteabot
b97dbf7a9e
Clarify the logger's MODE config option (#26267) (#26281)
Backport #26267 by @wxiaoguang

1. Fix the wrong document (add the missing `MODE=`)
2. Add a more friendly log message to tell users to add `MODE=` in their
config

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit a758337046)
2023-08-21 07:22:16 +02:00
Earl Warren
20557c6bdb
[BRANDING] define the forgejo webhook type
templates/swagger/v1_json.tmpl updated with `make generate-swagger`

(cherry picked from commit 88899c492e)
(cherry picked from commit 7171bd9617)
(cherry picked from commit 1a742446c1)
(cherry picked from commit d7c189d7b2)

Conflicts:
	routers/web/web.go
(cherry picked from commit cbdea868e4)
(cherry picked from commit 6cd150483b)
(cherry picked from commit 47246da8d3)
(cherry picked from commit f2aa0e6b76)
(cherry picked from commit 5a4fc69a16)
(cherry picked from commit 48e444ca09)
(cherry picked from commit 888e537811)
(cherry picked from commit 5121f493c9)
(cherry picked from commit 9394e55fdf)
(cherry picked from commit 3a2ce51768)
(cherry picked from commit 719ead3a65)
(cherry picked from commit 83e6f82e2a)
(cherry picked from commit 494a429b21)
(cherry picked from commit 4d775db6b4)
(cherry picked from commit b68f777dc2)
(cherry picked from commit 5b934023fa)
(cherry picked from commit 3b1ed8b16c)
(cherry picked from commit 6bc4a46c9f)
(cherry picked from commit 8064bb24a3)

Conflicts:
	templates/admin/hook_new.tmpl
	templates/org/settings/hook_new.tmpl
	templates/repo/settings/webhook/base_list.tmpl
	templates/repo/settings/webhook/new.tmpl
	templates/user/settings/hook_new.tmpl
	https://codeberg.org/forgejo/forgejo/pulls/1181

(cherry picked from commit 55f5588a91)

Conflicts:
	routers/web/web.go
	https://codeberg.org/forgejo/forgejo/issues/1219
2023-08-21 07:22:16 +02:00
Earl Warren
c862cc15c8
Revert "[BRANDING] define the forgejo webhook type"
This reverts commit 02ba08ca84.
2023-08-21 07:22:16 +02:00
Giteabot
1d900bc6a9
Avoid writing config file if not installed (#26107) (#26113)
Backport #26107 by @wxiaoguang

Just like others (oauth2 secret, internal token, etc), do not generate
if no install lock

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit e2596b0a99)
(cherry picked from commit 78722734fe)
2023-08-21 07:22:15 +02:00
Gusted
82cb19649d
[GITEA] Fix media description render for orgmode
- Backport of #1224
- In org mode you can specify an description for media via the following
syntax `[[description][media link]]`. The description is then used as
title or alt.
- This patch fixes the rendering of the description by seperating the
description and non-description cases and using `org.String()`.
- Added unit tests.
- Inspired by 6eb20dbda9/org/html_writer.go (L406-L427)
- Resolves https://codeberg.org/Codeberg/Community/issues/848
2023-08-12 16:04:33 +02:00
Earl Warren
a1986507b7
Revert "Avoid writing config file if not installed (#26107) (#26113)"
This reverts commit 78722734fe.

It does not create `LFS_JWT_SECRET` if `INSTALL_LOCK` is true and the
value of `LFS_JWT_SECRET` found in `app.ini` is incorrect. As a result
LFS_JWT_SECRET will not be set at all and the Forgejo admin will not
be notified that the value in the `app.ini` was ignored.
2023-07-30 09:30:36 +02:00
Giteabot
268569b462
Fix allowed user types setting problem (#26200) (#26206)
Backport #26200 by @lunny

Fix #25951

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
(cherry picked from commit 499c5594c3)
2023-07-30 07:46:19 +02:00
Giteabot
5a4b19435d
Calculate MAX_WORKERS default value by CPU number (#26177) (#26183)
(cherry picked from commit 892e24aaf1)
2023-07-30 07:46:18 +02:00
Lunny Xiao
28f4029e40
Display deprecated warning in admin panel pages as well as in the log file (#26094) (#26154)
backport #26094
Temporily resolve #25915
Related #25994

This PR includes #26007 's changes but have a UI to prompt administrator
about the deprecated settings as well as the log or console warning.
Then users will have enough time to notice the problem and don't have
surprise like before.

<img width="1293" alt="图片"
src="https://github.com/go-gitea/gitea/assets/81045/c33355f0-1ea7-4fb3-ad43-cd23cd15391d">

(cherry picked from commit c598741f01)
2023-07-30 07:42:53 +02:00
Earl Warren
f4fcdaba8c
Revert "[GITEA] do not use deprecatedSettingFatal for cosmetic reasons"
This reverts commit 2de8602855.
2023-07-30 07:42:38 +02:00
Giteabot
f4a8f10f64
Fix incorrect router logger (#26137) (#26143)
Backport #26137 by @wxiaoguang

A low-level mistake:

* `log.Info` is global `Info` function, which calls "default" logger
* `logger.Info` is the for router's logger

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 782b137682)
2023-07-26 13:49:15 +02:00
Giteabot
78722734fe
Avoid writing config file if not installed (#26107) (#26113)
Backport #26107 by @wxiaoguang

Just like others (oauth2 secret, internal token, etc), do not generate
if no install lock

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit e2596b0a99)
2023-07-26 13:49:15 +02:00
Giteabot
4be3270e87
Fix handling of Debian files with trailing slash (#26087) (#26098)
Backport #26087 by @KN4CK3R

Fixes #26022

- Fix handling of files with trailing slash
- Fix handling of duplicate package file errors
- Added test for both

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit a424f6d4f8)
2023-07-26 13:49:15 +02:00
Giteabot
942d02f8e9
Use stderr as fallback if the log file can't be opened (#26074) (#26083)
Backport #26074 by @wxiaoguang

If the log file can't be opened, what should it do? panic/exit? ignore
logs? fallback to stderr?

It seems that "fallback to stderr" is slightly better than others ....

(cherry picked from commit 8d9193680d)
2023-07-26 13:49:15 +02:00
Giteabot
8e4f521381
Adding remaining enum for migration repo model type. (#26021) (#26034)
Backport #26021 by @puni9869

Fixes: https://github.com/go-gitea/gitea/issues/26010

Adding remaining enum for migration repo model type.

Co-authored-by: puni9869 <80308335+puni9869@users.noreply.github.com>
(cherry picked from commit 8b002b429d)
2023-07-24 07:59:10 +02:00
Giteabot
d4fa6a846a
Fix env config parsing for "GITEA____APP_NAME" (#26001) (#26013)
Backport #26001 by @wxiaoguang

Regression of #24832

Fix the bug and add a test for it

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 4d5e3b9372)
2023-07-24 07:59:10 +02:00
Giteabot
f890856cf4
Update path related documents (#25417) (#25982)
Backport #25417 by @wxiaoguang

Update WorkPath/WORK_PATH related documents, remove out-dated
information.

Remove "StaticRootPath" on the admin config display page, because few
end user really need it, it only causes misconfiguration.

![image](https://github.com/go-gitea/gitea/assets/2114189/8095afa4-da76-436b-9e89-2a92c229c01d)

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit ee47face12)
2023-07-24 07:58:56 +02:00
wxiaoguang
8630d63f57
Avoid creating directories when loading config (#25944) (#25957)
Backport #25944

The "creating dir/file during load config" is a longstanding and complex
problem.

This PR only does a quick patch, it still needs more refactorings in the
future.

Fix #25938

(cherry picked from commit 5c3662b902)
2023-07-24 07:58:56 +02:00
KN4CK3R
7ad63bc41e
Disallow dangerous URL schemes (#25960) (#25964)
Regression: https://github.com/go-gitea/gitea/pull/24805
Closes: #25945

- Disallow `javascript`, `vbscript` and `data` (data uri images still
work) url schemes even if all other schemes are allowed
- Fixed older `cbthunderlink` tests

---------

Co-authored-by: delvh <dev.lh@web.de>
(cherry picked from commit ab54310731)
2023-07-24 07:58:56 +02:00
Giteabot
d1ed7ea9e3
Skip unuseful error message in dev mode when watching local filesystem (#25919) (#25927)
Backport #25919 by @wxiaoguang

Before, in dev mode, there might be some error logs like:

```
2023/07/17 13:54:51 ...s/assetfs/layered.go:221:WatchLocalChanges() [E] Unable to watch directory .: lstat /data/work/gitea/custom/templates: no such file or directory

```

Because there is no "custom/templates" directory.

After: ignore such error, no such error message anymore.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 9369b38315)
2023-07-24 07:58:56 +02:00