Add Ubuntu Policy Manual article

[BAMF0]

Description

This pull request will add the Ubuntu Policy Manual to the Ubuntu project docs in the form of the delta between the current Debian Policy Manual.

Important changes to the published Ubuntu Policy Manual:

• Omit sections considered entirely shared with Debian and link to corresponding web page, shown as "Shared with Debian, see Debian Policy Manual Chapter/Section/Appendix <link>"
• Add "Modifies" where the delta is sufficiently different to be noteworthy or warrant review.
• Add "Editor's note:" to areas that are outdated, need clarification, and/or re-review.
• Add Modifications area to the Copyright Notice (this should be changed according to what is preferred for the
  project).

---

Related issue

• Fixes: Publish Ubuntu Policy #333

---

Checklist

• I have read and followed the Ubuntu Project contributing guide
• My pull request is linked to an existing issue (if applicable)
• I have tested my changes, and they work as expected

---

Additional notes (optional)

Footnotes still use their original absolute numbering as in the Ubuntu Policy Manual package, i.e. new chapters will start counting from the previous footnote within the reStructuredText file. This is to more easily cross-reference footnotes from the original.

Some sections labeled "Modifies" may seem obviously outdated and are to be considered unnecessary delta. These should be thought of as a motivation for re-review if it actually warrants a delta, otherwise these can trivially be considered shared with Debian.

[BAMF0]

Also, sorry for the 3,000 lines addition! I did not think it reasonable to split it into multiple PRs as of the reference dependencies between chapters and sections.

[rkratky]

@mwhudson, @basak, @seb128, please, have a look at this addition. The PR introduces the Ubuntu Policy to this docs set as agreed upon in #devel:ubuntu.com. @BAMF0 has done an awesome job of separating the Ubuntu delta from the last published version of the Policy and then structuring it into an easily navigable document with links to the Debian Policy where the content is the same. (Not to mention the conversion from XML/HTML.)

I believe this will allow for a much easier maintenance and updates (and it will make the Policy actually available outside of the ubuntu-policy package).

Provided you're OK with the way this looks (preview build), let me know how restricted you'd like future modifications of this content. Should the Technical Board be the owner?

[basak]

Ack. The approach here seems reasonable to me, but it will take a long time to review!

[rkratky]

@BAMF0, it just occurred to me that this should link to https://documentation.ubuntu.com/project/how-ubuntu-is-made/concepts/debian-policy/ (and vice versa).

[mwhudson]

This is mostly a collection of thoughts and side comments than a review, I'll try to work our way through the more substantive bits eventually! Thanks so much for starting this!

[mwhudson]

I don't know if this is 100% the right thing to say, either in the sense of it being true in practice or if we want it to be true in practice. OTOH I'm not sure I know who to put here instead! Ideally we'd have an "ubuntu policy" group but I guess the fallback is the techboard. Maybe I can ask the TB about this this week.

[mwhudson]

This is all news to me :-)

[teward]

The proper solution would be to use update-maintainer from ubuntu-dev-tools which handles all this anyways. This is also, I believe, not necessarily actual policy. We should evaluate what ubuntu-dev-tools and update-maintainer scripts do before accepting this part.

[basak]

See also: https://wiki.ubuntu.com/DebianMaintainerField

[mwhudson]

til this mailing list exists

[mwhudson]

I don't think priority standard/optional/extra makes any real difference in Ubuntu

[mwhudson]

Certainly this language is not relevant to Ubuntu

[mwhudson]

I don't see how this is different from Debian...

[BAMF0]

Ah yes you are right, the difference was that the Debian article contained extra information, but now in hindsight they say exactly the same thing.

[mwhudson]

I'm sure that URL isn't right :-)

[teward]

It's not, it should probably be removed if there's no equivalent for Ubuntu.

[mwhudson]

In general for the sections that modify the Debian policy section it would be great to indicate how

[BAMF0]

That's a good point. For now, I think that most sections that are described as modifying Debian are really out-of-sync rather than the "modification" being an intentional change. Having the modifications as a bulleted list of some kind might be a good idea?

[mwhudson]

I suspect yes most differences are just us being behind. Maybe we can make the sections like this somehow and try to work our way through deciding whether we want to just take the updates? (which I suspect we will, mostly)

[mwhudson]

Somewhere we should talk about how all packages in Ubuntu are team maintained I guess.

[mwhudson]

Ah OK we can ignore one of my previous comments then :-)

[rkratky]

@mwhudson, thanks for looking through it.

I'd like to suggest that the review for this PR be limited to the publishing approach and related aspects (such as the manner of cross-linking the Debian Policy).

All other content concerns can (and should, I believe) be dealt with after this PR is merged. Two reasons:

• Separating questions of the publishing mechanics from the review of the content itself would allow us to move faster. Afterwards, individual parts of the Ubuntu Policy that might need modifications can be considered one by one.
• The Ubuntu Policy hasn't been touched in over a decade, but it has always been "published" (available from the ubuntu-policy package). Therefore, publishing it as it is now wouldn't constitute a regression. Granted, parts of it are likely obsolete, but fixing the Policy itself is out of scope for this PR.

cc @basak

[mwhudson]

I think the only change I would like to see before merging is some kind of annotation as to when a section that diverges from debian is simply where debian has changed and we have not followed or where there is some kind of ubuntu delta. Is that possible to achieve without too much tedium?

[rkratky]

some kind of annotation as to when a section that diverges from debian is simply where debian has changed and we have not followed or where there is some kind of ubuntu delta.

@mwhudson, I'd like to push back against that request (on Simon's behalf ;-)) This would go significantly beyond the scope of this task (which was to get the Ubuntu Policy -- in its current state -- from the forgotten ubuntu-policy package to this docs set). Actually engaging with the content itself and figuring out what needs to be updated and where would be asking for much more work.

I realize quite a lot more effort will need to go into the Ubuntu Policy in the future to make it usable and on par with the Debian counterpart again. But that really should be dealt with in subsequent PRs.

My suggestion would be to file a new issue describing (potential) next steps. As it is, the work that Simon has done already constitutes a huge improvement over the current state of affairs, so I think it would be fair to merge it.

[teward]

FYI as a member of the TB I think parts of this need revised before we can even approach 'merge approval' state. Things in the policy are not accurate or need updated (and have had no movement on the comments for > 2 weeks).

[mwhudson]

At the TB meeting today we agreed to spend some time looking at this over the next week or so -- we don't want to leave you and Simon hanging forever. But also this is an area where care is required!

[BAMF0]

At the TB meeting today we agreed to spend some time looking at this over the next week or so -- we don't want to leave you and Simon hanging forever. But also this is an area where care is required!

That sounds reasonable! Given the scale, I have waited with addressing comments until we have a notion of what is essential for an initial publishing. As Robert pointed out, I think it would be a good idea to consider how we can split this up more easily. Some parts might need rewriting for example, and introducing rewrites in this PR may not give them the proper attention for review.

[rkratky]

FYI as a member of the TB I think parts of this need revised before we can even approach 'merge approval' state. Things in the policy are not accurate or need updated.

@teward, I'm not disputing the need to update the content. My point in suggesting to merge now is that the Policy has -- officially -- been out there and in this outdated state for over 12 years. So, merging this PR wouldn't be a step back in that regard. But it would allow subsequent work on it to be done with the relative comfort of Git(Hub).

(and have had no movement on the comments for > 2 weeks)

Yes, my proposal would be to address those comments in follow up PRs that the TB could own. Given that we (Simon, myself) do not know what and how should be modified, nor do we have the authority to do so, it leads to this rather inefficient process where TB members would leave comments basically saying "edit this, edit that".

At the TB meeting today we agreed to spend some time looking at this over the next week or so -- we don't want to leave you and Simon hanging forever. But also this is an area where care is required!

@mwhudson, thank you. My concern is that while the actual content of the Policy is being dealt with in this PR, the onus is on Simon to make modifications. Correcting the Policy content and bringing it up-to-date is not what Simon signed up for ;) Once this is merged, the CODEOWNERS would be modified to make the TB the sole owners of the content, which would allow you to work on getting the content into a desired state without having to go through an intermediary.

[seb128]

Hey @BAMF0, thanks again for the work you have put into these changes, and sorry for the delay in reviewing them.

We discussed this PR during the TB meeting earlier this week (you can find the details of the conversation in the log).

The situation is currently blocked because the changes are complex to review, and everyone's bandwidth is limited right now. We feel it is going to be difficult to make progress on what is currently proposed, so we would like to suggest an alternative approach to help move things forward.

Our proposal is to:

• Withdraw the change as currently proposed.
• Start from the current ubuntu-policy package.
• Review the delta compared to the Debian Policy version it's based on and split it into logical changesets/commits (following the process documented in Ubuntu merge documentation).
• Start converting those changesets into GitHub PRs, opening one pull request per commit.

This approach should give us reviewable units and allow for easier iteration.

Additionally, we would like to propose that the Ubuntu Policy not try to copy the Debian sections. Instead, it should take a form similar to: Ubuntu Policy is based on the Debian policy (which can be found here <ref>), on top of which we apply the following changes, and then only document our specific modifications. This should make maintenance much easier and avoid an ongoing re-basing effort.

Let me know what you think. I'm also happy to have a meeting to discuss the proposal if you'd prefer to discuss this on a call.

[BAMF0]

Thanks for the response @seb128! No worries for the delay, with the coming release and all there are definitely other priorities on the table.

I read through the TB meeting log and think I understand the rationale behind your proposal. You are of course the owners and have final say, but I do not really think this is the best way to go about it. I considered the approach suggested at first, however by comparing it to the Debian Policy version it is based on, we do not reflect on what has changed since it's publication. What if there is new delta for instance? Would only listing the old differences encourage this line of thought?

One key detail to point out with this PR is that it, by itself, does not introduce any intentional changes to the Ubuntu Policy. The changes are instead indirect as a dependency on the current Debian Policy, and are in that case not contained within this document. Furthermore it is worth considering that the article's current appearance is in no way to be considered final, it is rather an initial draft for aiding reiteration. Copying Debian sections for instance, is done with the intention of rethinking whether there is something that Ubuntu should differ from in those subject areas. I absolutely agree that we should not duplicate these in this manner for a more "official" publication.

I would instead propose that this PR is included as a draft, and we could in that case remove links to it until it is deemed ready. You know more of course, but the workflow I would suggest is to:

• Review the policy section by section as they are separated by files. Subsequent PRs would then be targeted against a significantly smaller number of files (or even one at a time).
• If we accept Debian's version of a section, we can safely remove the duplicate.
• If not, prior intentional modifications are already marked with Ubuntu: and these sections should be re-reviewed. As I mentioned in the PR description "Modifies" is simply meant to highlight that the delta between the old and current policy is significant enough to warrant a re-review.

In other words, this workflow should be with the aim to review what we currently say, what we want to say, and strip away what is unnecessary to keep. I would be happy to discuss this over a call as well! I think that would be a bit easier probably.

[seb128]

I'm just going to reply to that specific point for now

however by comparing it to the Debian Policy version it is based on, we do not reflect on what has changed since it's publication. What if there is new delta for instance? Would only listing the old differences encourage this line of thought?

Comparing to the policy version it's currently based on would only to split and document the delta we had at the time. You can see it as a standard merge with Debian/rebase exercise. The first step is document our changes, then next one is to rebase which is the time where was ask the question of whether the change "still apply / needs to be reworked / can be dropped".

Or said different, each of those commits would lead to a PR which we would review in the lens of "let's look at what the current Debian policy says about the topic and evaluate if that delta still makes sense"

[BAMF0]

Or said different, each of those commits would lead to a PR which we would review in the lens of "let's look at what the current Debian policy says about the topic and evaluate if that delta still makes sense"

Edit: Reread, scrap my assumption about the ubuntu-policy merge proposal.

Yes exactly, I definitely agree! I think this is a first good step. However, I assume that this would still be a merge proposal against the ubuntu-policy package? In that case, the work of publishing it as an article (which is the point of this PR) would instead be deferred to the future.

A point I am not entirely understanding is: what is preventing a similar process for the contents of this PR? Perhaps it is rather where we want the "ground truth" for the policy to be? I am certain we would be able to derive and document the specific changes we had at the time in addition to this PR in order to aid in revising the policy. If only the old Ubuntu specific modifications are considered, wouldn't the task of finding new delta be even more insurmountable using that approach?

[rkratky]

@seb128 Thanks for the context and details (as well as sharing the conversation log). I believe both @BAMF0 and I now understand better your concerns.

As a preface, I'd like to clarify that we're not trying to sidestep the review process with this PR. The intention was to pull the content, as is, from the package, provide all the scaffolding needed for proper review/updates in the future (refs to Debian Policy etc.), and then leave it to the TB to do as it would.

---

To sum it up, from the log of the meeting, I gather there seem to be two main problems with this PR:

• By publishing the Policy in this way, its outdated state would be way more visible than it is now in the form of the package, which would put undue pressure on the TB to do the updates sooner rather than later -- something there's no capacity for.

• A separate concern is that given the size of the PR/diff, it's hard/impossible to see if the contents of the package is being reproduced here faithfully. Either through malicious intent or plain human error, the Policy may have been altered.

---

As for the first, I think we could do what you've discussed in the meeting: there's a "staging" area in the repo where we've been dumping things that went through initial conversion/migration but still needed work. The content in the staging area can be selectively published on the website (still outside of the main navig. structure), or it can stay hidden -- i.e. sit on main, ready to be hacked on, but not published.

I think this could be a good solution.

As for the second concern, I'm afraid we can't really address that without closing this and doing what @basak suggested, i.e. either creating a conversion script for you to review (instead of having to review the whole thing), or iterating slowly over small PRs.

Still, I'd like to highlight that Simon approached the work on this with exemplary diligence, so -- unless we'd be talking about intentional corruption of the content -- I'd say the chance of there being unaccounted changes is minimal.

---

Bottom line: the decision is yours to make, but I honestly don't think it would be fair to ask Simon or anyone else to do the PR-by-PR thing. If that's the way the TB would like to proceed, I'd propose just closing this PR and leaving it to the TB to deal with on their own (on a schedule that fits your expectation and requirements/capacity).

But I think it would be a pity to ditch the work Simon has done so far (though I understand the concerns raised).

Would the TB consider the "staging" area approach?

(One last thing: While I tried to explain why we were "neglecting" the comments on the specific details of the contents, I understand it may have looked as if we were just ignoring it. Would it make the "staging" approach acceptable if Simon and I addressed the specific comments left by @mwhudson and @teward first (with their input)? I realize that won't fix the Policy, but at least it could remove the problems that were deemed to be most glaring.)