与 firefox 的爱恨情仇

RISC-V
, , ,

起因

firefox 102 build 的时候会报错找不到 intersection 方法。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
101:24.74 error[E0599]: no method named `intersection` found for struct `FeatureFlags` in the current scope
101:24.74    --> servo/components/style/queries/feature.rs:126:14
101:24.74     |
101:24.74 103 | / bitflags! {
101:24.74 104 | |     /// Different flags or toggles that change how a expression is parsed or
101:24.74 105 | |     /// evaluated.
101:24.74 106 | |     #[derive(ToShmem)]
101:24.74 ...   |
101:24.74 120 | |     }
101:24.74 121 | | }
101:24.74     | |_- method `intersection` not found for this
101:24.74 ...
101:24.74 126 |           self.intersection(Self::CHROME_AND_UA_ONLY | Self::WEBKIT_PREFIX)
101:24.74     |                ^^^^^^^^^^^^ help: there is an associated function with a similar name: `intersects`
101:24.79 error[E0599]: no method named `intersection` found for struct `FeatureFlags` in the current scope
101:24.79    --> servo/components/style/queries/feature.rs:139:14
101:24.79     |
101:24.79 103 | / bitflags! {
101:24.79 104 | |     /// Different flags or toggles that change how a expression is parsed or
101:24.79 105 | |     /// evaluated.
101:24.79 106 | |     #[derive(ToShmem)]
101:24.79 ...   |
101:24.79 120 | |     }
101:24.79 121 | | }
101:24.79     | |_- method `intersection` not found for this
101:24.79 ...
101:24.79 139 |           self.intersection(Self::all_container_axes())
101:24.79     |                ^^^^^^^^^^^^ help: there is an associated function with a similar name: `intersects`
102:23.55 For more information about this error, try `rustc --explain E0599`.

报错的是servo/components/style/queries/feature.rs中的代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
bitflags! {
    /// Different flags or toggles that change how a expression is parsed or
    /// evaluated.
    #[derive(ToShmem)]
    pub struct FeatureFlags : u8 {
        /// The feature should only be parsed in chrome and ua sheets.
        const CHROME_AND_UA_ONLY = 1 << 0;
        /// The feature requires a -webkit- prefix.
        const WEBKIT_PREFIX = 1 << 1;
        /// The feature requires the inline-axis containment.
        const CONTAINER_REQUIRES_INLINE_AXIS = 1 << 2;
        /// The feature requires the block-axis containment.
        const CONTAINER_REQUIRES_BLOCK_AXIS = 1 << 3;
        /// The feature requires containment in the physical width axis.
        const CONTAINER_REQUIRES_WIDTH_AXIS = 1 << 4;
        /// The feature requires containment in the physical height axis.
        const CONTAINER_REQUIRES_HEIGHT_AXIS = 1 << 5;
    }
}

impl FeatureFlags {
    /// Returns parsing requirement flags.
    pub fn parsing_requirements(self) -> Self {
        self.intersection(Self::CHROME_AND_UA_ONLY | Self::WEBKIT_PREFIX)
    }

    /// Returns all the container axis flags.
    pub fn all_container_axes() -> Self {
        Self::CONTAINER_REQUIRES_INLINE_AXIS |
            Self::CONTAINER_REQUIRES_BLOCK_AXIS |
            Self::CONTAINER_REQUIRES_WIDTH_AXIS |
            Self::CONTAINER_REQUIRES_HEIGHT_AXIS
    }

    /// Returns our subset of container axis flags.
    pub fn container_axes(self) -> Self {
        self.intersection(Self::all_container_axes())
    }
}

这里面提供 intersection 方法的宏是 bitflags，在正常情况下该宏展开是有 intersection的，故猜测可能是 rust 稍微有点不对劲。

折腾 cargo expand

进行到这，想采用 cargo expand 工具来尝试展开宏，看看是宏展开有问题，还是其他的问题。
运行 cargo expand，提示错误:

1
2
3
cargo expand --lib main
ERROR: cannot expand single item (main) without rustfmt.
Install rustfmt by running `rustup component add rustfmt --toolchain nightly`.

查询 rustfmt

1
2
3
4
5
pacman -F rustfmt
extra/rust 1:1.62.0-1 [installed]
    usr/bin/rustfmt
community/rustup 1.25.1-1
    usr/bin/rustfmt

rust 里面是有rustfmt, 为啥cargo expand还需要通过 rustup 来装 rustfmt? 我不理解。同时感谢 c10s 发的 issus Can’t use with rustfmt installed not by rustup，虽然被光速关闭。

感谢乔老师： 乔老师 revert 了之前的 tootoolchain_find 的 patch,使得 cargo expand 能够使用 rust 的 rustfmt.

然后发现 cargo expand 是在自己代码里面找 nightly toolchain 调用的，然而目前各大发行版打的 rust 都是正式版的。根本没有 nightly 的 rust。

该方法暂时卡住。

被点醒

图片加载失败
图片加载失败

感谢大佬发现了我之前没发现的华点。

这个 patch 怎么这么大

发现 bitflags产生的问题后，就开始解决。顺便跟着 makotokato新的 pr 来更新一下旧版的 patch。

诶，更新了 midir， 更一下，更一下。patch 达到了惊人了 2G。

1
2
[eastdong@East Downloads]$ ls -l makotokato-riscv64-support-and-zenithal-backported.patch
-rw-r--r-- 1 eastdong eastdong 238161059 Aug 28 15:56 makotokato-riscv64-support-and-zenithal-backported.patch

这么大，不太对劲。看看这个 commit, ./mach vendor rust , 更新了很多库文件。

查看 firefox 文档，发现可以使用 ./mach vendor 来更新库文件。这条命令会读取在 Cargo.lock 中写的软件版本，从网络上下载库。同时还会检查目录下依赖版本的冲突情况，自动替换为合适的依赖版本。

./mach vendor rust

初次使用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
The details of the failure are as follows:

mozversioncontrol.InvalidRepoPath: Unknown VCS, or not a source checkout: /build/firefox/src/firefox-104.0.1

  File "/build/firefox/src/firefox-104.0.1/python/mozbuild/mozbuild/vendor/mach_commands.py", line 200, in vendor_rust
    ok = vendor_command.vendor(**kwargs)
  File "/build/firefox/src/firefox-104.0.1/python/mozbuild/mozbuild/vendor/vendor_rust.py", line 596, in vendor
    if not ignore_modified and self.has_modified_files():
  File "/build/firefox/src/firefox-104.0.1/python/mozbuild/mozbuild/vendor/vendor_rust.py", line 198, in has_modified_files
    for f in self.repository.get_changed_files("M")
  File "/build/firefox/src/firefox-104.0.1/python/mozbuild/mozbuild/util.py", line 1081, in __get__
    setattr(instance, name, self.func(instance))
  File "/build/firefox/src/firefox-104.0.1/python/mozbuild/mozbuild/base.py", line 445, in repository
    return get_repository_object(self.topsrcdir)
  File "/build/firefox/src/firefox-104.0.1/python/mozversioncontrol/mozversioncontrol/__init__.py", line 700, in get_repository_object
    raise InvalidRepoPath(f"Unknown VCS, or not a source checkout: {path}")

./mach vendor rust 会通过读取 VCS 来判断是否在开发环境。但是可以使用 --ignore-modified 来跳过检查。

经过测试，在在 git 仓库里跑 mach vendor rust 是可以运行的

./mach vendor rust --ignore-modified

1
2
3
4
5
6
$./mach vendor rust --ignore-modified

error: no such subcommand: `vet`

        Did you mean `fmt`?

读一下 vendor rust 的源代码。找到 vet 的定义

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
# Only emit warnings for cargo-vet for now.
env = os.environ.copy()
env["PATH"] = os.pathsep.join(
    (
        str(Path(cargo).parent),
        os.environ["PATH"],
    )
)
flags = ["--output-format=json"]
if "MOZ_AUTOMATION" in os.environ:
    flags.append("--locked")
    flags.append("--frozen")
res = cargo_vet(
    self,
    flags,
    stdout=subprocess.PIPE,
    env=env,
)
if res.returncode:
    vet = json.loads(res.stdout)
    logged_error = False
    for failure in vet.get("failures", []):
        failure["crate"] = failure.pop("name")
        self.log(
            logging.ERROR,
            "cargo_vet_failed",
            failure,
            "Missing audit for {crate}:{version} (requires {missing_criteria})."
            " Run `./mach cargo vet` for more information.",
        )
        logged_error = True
    # NOTE: This could log more information, but the violation JSON
    # output isn't super stable yet, so it's probably simpler to tell
    # the caller to run `./mach cargo vet` directly.
    for key in vet.get("violations", {}).keys():
        self.log(
            logging.ERROR,
            "cargo_vet_failed",
            {"key": key},
            "Violation conflict for {key}. Run `./mach cargo vet` for more information.",
        )
        logged_error = True

我们可以看到 vet 是其实是将 res 的 stdout 编码为 python 对象。

res = cargo_vet() ,阅读前面的代码 from mozbuild.mach_commands import cargo_vet确定 cargo_vet 是由 mach_commands 引入的。

根据 mach_commands.py中的代码。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
def cargo_vet(command_context, arguments, stdout=None, env=os.environ):
    from mozbuild.bootstrap import bootstrap_toolchain

    # Logging of commands enables logging from `bootstrap_toolchain` that we
    # don't want to expose. Disable them temporarily.
    logger = logging.getLogger("gecko_taskgraph.generator")
    level = logger.getEffectiveLevel()
    logger.setLevel(logging.ERROR)

    env = env.copy()
    cargo_vet = bootstrap_toolchain("cargo-vet")
    if cargo_vet:
        env["PATH"] = os.pathsep.join([cargo_vet, env["PATH"]])
    logger.setLevel(level)

可以简单的理解为通过 bootstrap_toolchain 来 build 一个 cargo-vet 并将其加入到 PATH 中。然后就卡住了。

研究了一下 bootstrap_toolchain 的代码没有找到很好的解决方案。

更换思路

既然我们没办法解决找不到 vet ，那我们可以尝试解决调用 vet 的函数。

./mach vendor rust --ignore-modified 有 VCS 状态下

经过测试，在 git 仓库里跑 mach vendoro rust --ignre-modified 没问题。这就是有 vcs 的状态

解决方案

阅读 vendor rust 代码，发现 cargo_vet 的作用是验证第三方库文件是受信任的。

既然找不到 vet ，那我们可以注释所有和 vet 相关代码，并且返回 True 即可使得编译正常运行。

感谢 gentoo 小队的大佬提供的解决方案。具体代码在 https://github.com/gentoo/riscv/blob/master/www-client/firefox/files/firefox-riscv64-hack.patch 。