Commit graph

2841 commits

Author SHA1 Message Date
6543
f832e8eeea
Fix some API bugs (#16184) (#16190)
* Fix some API bugs (#16184)

* Repository object only count releases as releases (fix #16144)

* EditOrg respect RepoAdminChangeTeamAccess option (fix #16013)

* adjut to v1.14
2021-06-18 19:47:34 +08:00
zeripath
544ef7d394
Encrypt migration credentials at rest (#15895) (#16187)
Backport #15895

Storing these credentials is a liability.

* Encrypt credentials with SECRET_KEY before persisting to task queue table (they need to be persisted due to the nature of the task queue)
  - security in depth: helps when attacker has access to DB only, but not app.ini
* Delete all credentials (even encrypted) from the task table, once the migration is done, for safety
  - security in depth: minimizes leaked data if attacker gains access to snapshot of both DB and app.ini
2021-06-17 22:59:28 +02:00
zeripath
5ff807acde
Run processors on whole of text (#16155) (#16185)
Backport #16155

There is an inefficiency in the design of our processors which means that Emoji
and other processors run in order n^2 time.

This PR forces the processors to process the entirety of text node before passing
back up. The fundamental inefficiency remains but it should be significantly
ameliorated.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-06-17 20:01:33 +02:00
zeripath
849d316d8d
issue-keyword class is being incorrectly stripped off spans (#16163) (#16172)
Backport #16163

Bluemonday sanitizer regexp rules are not additive, so the addition of the icons,
emojis and chroma syntax policy has led to this being stripped.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-06-16 06:35:54 -04:00
zeripath
f034804e5d
Set self-adjusting deadline for connection writing (#16068) (#16123)
In #16055 it appears that the simple 5s deadline doesn't work for large
file writes. Now we can't - or at least shouldn't just set no deadline
as go will happily let these connections block indefinitely. However,
what seems reasonable is to set some minimum rate we expect for writing.

This PR suggests the following algorithm:

* Every write has a minimum timeout of 5s (adjustable at compile time.)
* If there has been a previous write - then consider its previous
deadline, add half of the minimum timeout + 2s per kb about to written.
* If that new deadline is after the minimum timeout use that.

Fix #16055

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-06-11 00:26:32 +03:00
6543
ac84bb7183
Fix data URI scramble (#16098) (#16118)
* Fix data URI scramble (#16098)

* Removed unused method.

* No prefix for data uris.

* Added test to prevent regressions.

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2021-06-09 16:31:40 +02:00
6543
2265058c31
git migration: don't prompt interactively for clone credentials (#15902) (#16082)
* don't prompt interactively for clone credentials

* apply GIT_TERMINAL_PROMPT=0 to all git cmds

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>

Co-authored-by: Norwin <noerw@users.noreply.github.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-06-06 14:02:34 +08:00
Jimmy Praet
8007602b40
Don't manipulate input params in email notification (#16011) (#16033)
Backport #16011
2021-05-31 02:17:34 -04:00
techknowlogick
3a79f1190f
Fix setting of SameSite on cookies (#15989) (#15991)
Fix #15972

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-05-27 10:02:39 -04:00
zeripath
159bc8842a
Restore PAM user autocreation functionality (#15825) (#15867)
Backport #15825

* Restore PAM user autocreation functionality

PAM autoregistration of users currently fails due to email invalidity.
This PR adds a new setting to PAM to allow an email domain to be set
or just sets the email to the noreply address and if that fails falls
back to uuid@localhost

Fix #15702

Signed-off-by: Andrew Thornton <art27@cantab.net>

* As per KN4CKER

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-05-19 10:42:36 -04:00
6543
8c4bf4c3b4
GitHub: migrate draft releases too (#15884) (#15888)
* GitHub: migrate draft releases too

* refactor
2021-05-16 09:24:28 +02:00
6543
3bcf2e5c18
Close the gitrepo when deleting the repository (#15876) (#15887)
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-05-16 00:45:17 +03:00
zeripath
41422f0df0
Add timeout to writing to responses (#15831) (#15872)
Backport #15831

In #15826 it has become apparent that there are a few occasions when a response can
hang during writing, and because there is no timeout go will happily just block
interminably. This PR adds a fixed 5 second timeout to all writes to a connection.

Fix #15826

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-14 17:42:27 +01:00
KN4CK3R
f773733252
Fix LFS commit finder not working (#15856) (#15874)
* Create a copy of the sha bytes.

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-05-14 16:39:59 +01:00
zeripath
cbaf8e8785
Stop calling WriteHeader in Write (#15862) (#15873)
Backport #15862

Fixes http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)

* Looking again we don't need this writeHeader as all of our downstream
implementations will always do it for us

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-14 15:38:35 +01:00
zeripath
387a1bc472
fix truncate utf8 string (#15828) (#15854)
Backport #15828

* fix truncate utf8 string.

* revoke truncated user info.

Co-authored-by: yan <sxty32@gmail.com>
2021-05-13 16:10:29 +02:00
zeripath
a83cde2f3f
Tagger can be empty, as can Commit and Author - tolerate this (#15835) (#15839)
Backport #15835

Unfortunately some old repositories can have tags with empty Tagger, Commit
or Author. Go-Git variants will always have empty values for these whereas
the native git variant leaves them at nil. The simplest solution is just to
always have these set to empty Signatures.

v156 migration also makes the incorrect assumption that these cannot be empty.
Therefore add some handling to this and add logging and adjust broken
logging elsewhere in this migration.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-05-12 20:09:16 +01:00
zeripath
332eb2f6d2
Queue manager FlushAll can loop rapidly - add delay (#15733) (#15840)
Backport #15733

* Queue manager FlushAll can loop rapidly - add delay

Add delay within FlushAll to prevent rapid loop when workers are busy

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per lunny

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-12 18:48:11 +01:00
John Olheiser
d054c4e7f3
Add err to log (#15813) (#15824)
Signed-off-by: jolheiser <john.olheiser@gmail.com>
2021-05-10 16:38:37 -04:00
Lunny Xiao
5e562e9b30
Move restore repo to internal router and invoke from command to avoid open the same db file or queues files (#15790) (#15816)
* Move restore repo to internal router and invoke from command to avoid open the same db file or queues files

* Follow @zeripath's review

* set no timeout for resotre repo private request

* make restore repo cancelable
2021-05-10 21:14:59 +08:00
6543
c57e908f36
Tests should use test files (#15801) (#15806) 2021-05-10 01:39:14 +08:00
6543
d644709b22
Exponential Backoff for ByteFIFO (#15724) (#15793)
This PR is another in the vein of queue improvements. It suggests an
exponential backoff for bytefifo queues to reduce the load from queue
polling. This will mostly be useful for redis queues.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-05-08 14:27:00 -04:00
6543
22d700edfd
Set GIT_DIR correctly if it is not set (#15751) (#15769)
* Set GIT_DIR correctly if it is not set

* Expand out templates

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
2021-05-07 20:01:25 +02:00
zeripath
6782a64a4a
Defer closing the gitrepo until the end of the wrapped context functions (#15653) (#15746)
* Defer closing the gitrepo until the end of the wrapped context functions (#15653)

Backport #15653

There was a mistake in #15372 where deferral of gitrepo close occurs before it should.

This PR fixes this.
2021-05-07 18:28:02 +02:00
zeripath
1ec11ac87e
Drop back to use IsAnInteractiveSession for SVC (#15749) (#15762)
Backport #15749

* Drop back to use IsAnInteractiveSession for SVC

There is an apparent permission change problem when using
IsWindowsService to determine if the SVC manager should be
used.

This PR simply drops back to using IsAnInteractiveSession as
this does not change behaviour.

Fix #15454

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Yes staticcheck I know this is deprecated

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Just leave me alone lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-05-07 17:44:35 +02:00
6543
2c2a30d6bb
Fix bug where repositories appear unadopted (#15757) (#15767)
Fix bug where repositories with capital letters in their names appear unadopted.

Fix #15755

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-05-07 17:07:39 +02:00
a1012112796
a83fb3a83a
fix some ui bug about draft release (#15137) (#15745)
* fix some ui bug about draft release

- should not show draft release in tag list because
  it will't create real tag
- still show draft release without tag and commit message
  for draft release instead of 404 error
- remove tag load for attachement links because it's useless

Signed-off-by: a1012112796 <1012112796@qq.com>

* add test code

* fix test

That's because has added a new release in relaese test database.

* fix dropdown link for draft release
2021-05-06 21:23:26 +02:00
zeripath
462c6fdee2
Fix setting redis db path (#15698) (#15708)
Backport #15698

There is a bug setting the redis db in the common nosql manager whereby the db path
always fails.

This PR fixes this.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-03 22:30:30 +01:00
zeripath
4fa2804238
Performance improvement for last commit cache and show-ref (#15455) (#15701)
Backport #15455

* Improve performance when there are multiple commits in the last commit cache

* read refs directly if we can

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-03 16:51:41 +02:00
6543
a725d31496
Delete references if repository gets deleted (#15681) (#15684)
* Remove DeletedBranch and LFSLocks.

* Sort beans.

Co-authored-by: KN4CK3R <KN4CK3R@users.noreply.github.com>
Co-authored-by: zeripath <art27@cantab.net>
2021-05-01 00:09:58 +02:00
KN4CK3R
54263ff123
Delete protected branch if repository gets removed (#15658) (#15676)
* Added missing error parameters.

* Delete protected branch if repository gets removed.

* Added doctor fix.
2021-04-30 19:59:42 +01:00
6543
3bde297121
[API] pull notification subject status: add "merged" (#15344) (#15654)
Current subject status can be "", "open" and "closed". This add "merged" to it.
2021-04-28 20:24:56 +01:00
Lunny Xiao
cae46216e4
fix webhook timeout bug (#15613) (#15621)
* Also fix the potential problem in httplib
2021-04-26 14:42:12 +02:00
zeripath
6e2dacfef6
If the default branch is not present do not report error on stats indexing (follow-up of #15546) (#15583) (#15594)
Backport #15546
Backport #15583

 #15546 doesn't completely fix this problem because the error returned is an ObjectNotExist
error not a BranchNotExist error.

Add test for ErrObjectNotExist too

Fix #15257

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-04-22 22:30:18 +02:00
Lunny Xiao
c0869c295a
Fix lfs management find (#15537) (#15578)
* Fix lfs management find (#15537)

Fix #15236

* Do not do 40byte conversion within ParseTreeLine
* Missed a to40ByteSHA

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>

* Remove space

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-04-22 20:32:48 +02:00
zeripath
248b67af6f
Fix NPE on view commit with notes (#15561) (#15573)
Backport #15561

Fix #15558

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-04-21 15:11:43 -04:00
KN4CK3R
5da024a019
Add ETag header (#15370) (#15552)
* Add ETag header.

* Comply with RFC 7232.

* Moved logic into httpcache.go

* Changed name.

* Lint

* Implemented If-None-Match list.

* Fixed missing header on *

* Removed weak etag support.

* Removed * support.

* Added unit test.

* Lint

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-04-20 12:01:58 -04:00
Lunny Xiao
eff2499be7
Fix bug on commit graph (#15517) (#15530) 2021-04-17 14:46:30 +02:00
6543
729fa06468
migration: github: if rate limit is not enabled, ignore it (#15490) (#15495)
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-04-15 18:24:01 +03:00
zeripath
b228a0aa44
Use index of the supported tags to choose user lang (#15452) (#15488)
Backport #15452

Fix #14793.

The previous implementation used the first return value of matcher.Match, which is the chosen language tag but may contain extensions such as de-DE-u-rg-chzzzz.

As mentioned in the documentation of language package, matcher.Match also returns the index of the supported tags, so I think it is better to use it rather than manipulate the returned language tag.

Co-authored-by: Naohisa Murakami <tiqwab.ch90@gmail.com>
2021-04-15 16:47:43 +02:00
Lunny Xiao
9e7e11224f
Fix potential copy lfs records failure when fork a repository (#15441) (#15485) 2021-04-15 16:13:14 +02:00
zeripath
00e55dd223
Prevent superfluous response.WriteHeader (#15456) (#15476)
Backport #15456

This PR simply checks the status before writing the header.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-04-15 11:02:42 +01:00
zeripath
2c4f1ed13e
Fix ambiguous argument error on tags (#15432) (#15474)
Backport #15432

There is a weird gotcha with GetTagCommitID that because it uses git rev-list
can cause an ambiguous argument error.

This PR simply makes tags use the same code as branches.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-04-14 14:53:01 -04:00
6543
c3398906a1
use repo1_bare to test against (#15402) (#15404) 2021-04-11 19:48:35 +02:00
zeripath
1fe5fe419e
Always set the merge base used to merge the commit (#15352) (#15385)
Backport #15352

The issue is that the TestPatch will reset the PR MergeBase - and it is possible for TestPatch to update the MergeBase whilst a merge is ongoing. The ensuing merge will then complete but it doesn't re-set the MergeBase it used to merge the PR.

Fixes the intermittent error in git test.

Signed-off-by: Andrew Thornton art27@cantab.net
2021-04-10 14:08:30 +02:00
zeripath
67a12b8fac
Turn RepoRef and RepoAssignment back into func(*Context) (#15372) (#15377)
Backport #15372

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-04-09 22:24:40 -04:00
zeripath
53c2136a9a
Upgrade to bluemonday 1.0.7 (#15379) (#15380)
* Upgrade to bluemonday 1.0.7 (#15379)

Backport #15379

Fix #15349

Signed-off-by: Andrew Thornton <art27@cantab.net>

* resolve CI

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-04-09 19:41:30 -04:00
KN4CK3R
cf9b6c281f
Close file on invalid range (Addition to #15166) (#15268) (#15308)
* Close file on invalid range.

* Close on seek error

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Moved 'Seek' into server.

* io.ReadSeekCloser is only available in Go 1.16

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-04-06 15:25:31 -04:00
6543
a8c6a4a70e
Fix bug in Wrap (#15302) (#15309)
Whilst doing other work I have noticed that there is an issue with Wrap when passing an
http.Handler - the next should be the next handler in line not empty.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
2021-04-06 18:44:24 +02:00
6543
e6050e80f7
Update to bluemonday-1.0.6 (#15294) (#15297)
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-04-06 01:36:58 +01:00