Chinese Characters in Logs Appear as Garbled Text in Console for ASP.NET Projects

[sangyuxiaowu]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

When running an ASP.NET project using .NET Aspire, I have encountered an issue where Chinese characters in the logs appear as garbled text in the console. However, these characters are displayed correctly in the structured logs.

Expected Behavior

Chinese characters should be displayed correctly in the console output.

Steps To Reproduce

1. Create a new ASP.NET project using .NET Aspire.
2. Add logging for some operations, including messages with Chinese characters.
3. Run the project and observe the console output.
4. Compare the console output with the structured logs.

_logger.LogInformation("中文输出测试");

Exceptions (if any)

No response

.NET Version info

.NET SDK:
Version: 9.0.101
Commit: eedb237549
Workload version: 9.0.100-manifests.4a280210
MSBuild version: 17.12.12+1cce77968

运行时环境:
OS Name: Windows
OS Version: 10.0.22631
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\9.0.101\

已安装 .NET 工作负载:
[maui-windows]
安装源文件: VS 17.12.35527.113
清单版本: 9.0.0/9.0.100
清单路径: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.maui\9.0.0\WorkloadManifest.json
安装类型: Msi

[maccatalyst]
安装源文件: VS 17.12.35527.113
清单版本: 18.1.9163/9.0.100
清单路径: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.maccatalyst\18.1.9163\WorkloadManifest.json
安装类型: Msi

[ios]
安装源文件: VS 17.12.35527.113
清单版本: 18.1.9163/9.0.100
清单路径: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.ios\18.1.9163\WorkloadManifest.json
安装类型: Msi

[aspire]
安装源文件: VS 17.12.35527.113
清单版本: 8.2.2/8.0.100
清单路径: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.aspire\8.2.2\WorkloadManifest.json
安装类型: Msi

[android]
安装源文件: VS 17.12.35527.113
清单版本: 35.0.7/9.0.100
清单路径: C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.android\35.0.7\WorkloadManifest.json
安装类型: Msi

配置为在安装新清单时使用 loose manifests。

Host:
Version: 9.0.0
Architecture: x64
Commit: 9d5a6a9aa4

.NET SDKs installed:
9.0.101 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 9.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
Not set

global.json file:
Not found

Learn more:
https://aka.ms/dotnet/info

Download .NET:
https://aka.ms/dotnet/download

Anything else?

No response

[JamesNK]

@karolz-ms I did some testing it appears that DCP returns ? for Chinese and emoji characters returned from GetLogSubresource.

ReadSubResourceAsStreamAsync in the k8s client returns a Stream. A StreamReader is created to read the bytes to characters. No encoding is specified here so I believe it does auto detection, and is deciding to use UTF8.

Does Aspire need to do something differently here, or does DCP need changes to support non latin characters?

[JamesNK]

I got it to work by setting the output encoding of the web app to UTF8:

Console.OutputEncoding = Encoding.UTF8;

From docs:

The default code page that the console uses is determined by the system locale.

@sangyuxiaowu Can you let me know what the encoding name of the value set to Console.OutputEncoding is?

[karolz-ms]

@JamesNK do you still want me to take a look? Is DCP doing something wrong?

[JamesNK]

I think DCP might need some fixes here. I tested with an output encoding of Encoding.Unicode (UTF-16) and bad data is returned in the dashboard.

Console output when running the app directly:

Console output for the app in dashboard:

Content read from the DCP stream is bad so it is a problem before it reaches the app host:

[karolz-ms]

On DCP side we should consider checking for BOM at the beginning of the log stream, and/or check for 16-bit wide LF vs 8-bit wide LF as the line separator.

From the DCP client perspective we should standardize on UTF-8 to make things simple for Aspire app host.

[sangyuxiaowu]

I got it to work by setting the output encoding of the web app to UTF8:

Console.OutputEncoding = Encoding.UTF8;
From docs:

The default code page that the console uses is determined by the system locale.

@sangyuxiaowu Can you let me know what the encoding name of the value set to Console.OutputEncoding is?

Thank you all for your help and suggestions. I have tried setting Console.OutputEncoding = Encoding.UTF8 and confirmed that the encoding name is UTF-8. This solution works and resolves the issue with the Chinese characters displaying correctly in the console.

I appreciate your assistance in resolving this problem. Thank you!