linux负载均衡 和 系统负载分析笔记

1 负载均衡

1.1 计算负载

1.1.1 PELT算法简介

从Linux3.8内核以后进程的负载计算不仅考虑权重,⽽且跟踪每个调度实体的历史负载情况,该算法称为PELT(Per-entity Load Tracking)

                                《奔跑吧Linux内核》卷1:基础架构;P505

相关资料

  • https://lwn.net/Articles/531853/
  • http://www.wowotech.net/process_management/PELT.html
  • http://www.wowotech.net/process_management/pelt.html
  • http://www.wowotech.net/process_management/450.html

1.1.2 记录负载的数据结构struct sched_avg

1.1.2.1 定义
/** The load_avg/util_avg accumulates an infinite geometric series* (see __update_load_avg() in kernel/sched/fair.c).** [load_avg definition]**   load_avg = runnable% * scale_load_down(load)** where runnable% is the time ratio that a sched_entity is runnable.* For cfs_rq, it is the aggregated load_avg of all runnable and* blocked sched_entities.** [util_avg definition]**   util_avg = running% * SCHED_CAPACITY_SCALE** where running% is the time ratio that a sched_entity is running on* a CPU. For cfs_rq, it is the aggregated util_avg of all runnable* and blocked sched_entities.** load_avg and util_avg don't direcly factor frequency scaling and CPU* capacity scaling. The scaling is done through the rq_clock_pelt that* is used for computing those signals (see update_rq_clock_pelt())** N.B., the above ratios (runnable% and running%) themselves are in the* range of [0, 1]. To do fixed point arithmetics, we therefore scale them* to as large a range as necessary. This is for example reflected by* util_avg's SCHED_CAPACITY_SCALE.** [Overflow issue]** The 64-bit load_sum can have 4353082796 (=2^64/47742/88761) entities* with the highest load (=88761), always runnable on a single cfs_rq,* and should not overflow as the number already hits PID_MAX_LIMIT.** For all other cases (including 32-bit kernels), struct load_weight's* weight will overflow first before we do, because:**    Max(load_avg) <= Max(load.weight)*** Then it is the load_weight's responsibility to consider overflow* issues.*/
struct sched_avg {u64             last_update_time;u64             load_sum;u64             runnable_load_sum;u32             util_sum;u32             period_contrib;unsigned long           load_avg;unsigned long           runnable_load_avg;unsigned long           util_avg;struct util_est         util_est;
} ____cacheline_aligned;
1.1.2.2 struct sched_avg成员变量含义

1.1.3 数据结构组织关系

进程队列

进程调度实体

1.1.4 ___update_load_avg() 和 ___update_load_sum();

___update_load_avg():计算量化负载(load_avg) 和 实际算⼒(util_avg)。
___update_load_sum():计算工作负载

                                《奔跑吧Linux内核》卷1:基础架构;P515

                                《Linux内核深度解析》P104

1.1.5 查看单个进程的负载信息

 例如,查看pid为7202进程的负载信息

# cat /proc/7202/sched | grep se.avg
se.avg.load_sum                              :                   15
se.avg.runnable_sum                          :                15521
se.avg.util_sum                              :                15521
se.avg.load_avg                              :                    0
se.avg.runnable_avg                          :                    0
se.avg.util_avg                              :                    0
se.avg.last_update_time                      :       42221565865984
se.avg.util_est.ewma                         :                    8
se.avg.util_est.enqueued                     :                    8

1.1.6 查看公平队列(cfs_rq)的负载信息

# cat /sys/kernel/debug/sched/debug
......
cfs_rq[0]:/.......load_avg                      : 0  .runnable_avg                  : 1  .util_avg......
cfs_rq[1]:/.......load_avg                      : 2.runnable_avg                  : 6.util_avg......
cfs_rq[2]:/.......load_avg                      : 0.runnable_avg                  : 0.util_avg                      : 0......

1.1.7 中断处理程序占用的负载

需要打开内核配置:CONFIG_HAVE_SCHED_AVG_IRQ

1.2 完全公平调度类的负载均衡

1.2.1 调度域 和 调度组

1.2.1.1 简介

调度域实际上是⼀个CPU集合,它们的⼯作量应该由内核保持平衡。        《深⼊理解LINUX内核》P285

内核按照处理器拓扑层次划分调度域层次,每个调度域包含多个调度组。        《Linux内核深度解析》P100

调度组是负载均衡调度的最⼩单位。在最低层级的调度域中,通常⼀个调度组描述⼀个CPU。

调度域和调度组的关系。        《奔跑吧Linux内核》卷1:基础架构;P521

只有在某个调度域的某个组的总⼯作量远远低于同⼀个调度域的另⼀个组的⼯作量时,才把进程从⼀个CPU迁移到另⼀个CPU。

                        《深⼊理解LINUX内核》P285

1.2.1.2 调度域数据结构:struct sched_domain;

1.2.1.3 调度域的相关配置:/sys/kernel/debug/sched/domains/

/sys/kernel/debug/sched/domains/cpuX/domainX/目录下的内容实际上就是struct sched_domain的成员变量。

# tree /sys/kernel/debug/sched/domains/

1.2.1.4 查看调度域统计信息:/proc/schedstat

 linux内核调度相关操作发生的很频繁,所以记录调度相关的信息会带来一定的开销,默认情况下内核不会去记录这些信息。如果需要内核记录调度相关的信息,可以执行下面的命令:
        echo 1 > /proc/sys/kernel/sched_schedstats

执行完上面的命令,可以看到下面的信息:

 cat /proc/schedstat 
version 15
timestamp 4300757388
cpu0 17 0 301792 149974 155077 44875 517287834502 12966866224 2808579
domain0 11 41211 41126 49 85 36 0 0 41126 605 603 0 2 2 0 0 603 7133 6658 50 517 466 0 0 6657 0 0 0 0 0 0 0 0 0 16518 3737 0
domain1 ff 17882 17020 836 880 29 0 0 13754 61 60 1 1 0 0 0 15 6496 4341 1655 2337 607 6 1 4340 1 0 1 0 0 0 0 0 0 93684 20030 0
cpu1 0 0 387454 180994 225274 95619 472295220781 13216661039 2921559
domain0 22 57246 57149 43 89 47 0 0 57166 462 462 0 0 0 0 0 462 14118 13659 47 499 450 0 0 13657 0 0 0 0 0 0 0 0 0 17150 4400 0
domain1 ff 21251 20522 703 747 35 1 0 15691 52 51 0 2 2 0 0 8 13514 11220 1764 2475 620 4 2 11218 1 0 1 0 0 0 0 0 0 112505 22988 0
cpu2 1 0 293600 143123 138283 43027 541851523627 11551033606 2640733
domain0 44 39355 39274 33 73 40 0 0 39308 697 696 1 31 0 0 0 696 6899 6449 56 489 432 0 0 6449 0 0 0 0 0 0 0 0 0 13490 3002 0
domain1 ff 16536 15525 973 1031 36 1 1 12770 125 123 2 2 0 0 0 27 6301 4189 1555 2353 679 3 0 4189 2 0 2 0 0 0 0 0 0 81766 17801 0
cpu3 20 0 290491 141093 135912 43103 501320856499 11449769836 2678896
domain0 88 39419 39343 46 81 35 0 0 39353 643 642 1 55 0 0 0 642 6916 6509 42 427 385 0 0 6509 0 0 0 0 0 0 0 0 0 12498 2412 0
domain1 ff 16423 15442 943 1012 40 1 1 12678 112 109 2 3 1 0 0 19 6312 4269 1531 2334 648 5 0 4269 1 0 1 0 0 0 0 0 0 80311 17763 0
cpu4 1 0 301517 145953 147085 45129 425431559500 16937152347 3871699
domain0 11 39179 38999 76 132 49 0 0 38998 638 637 1 23 0 0 0 637 6276 5827 50 517 427 0 0 5790 0 0 0 0 0 0 0 0 0 14980 3230 0
domain1 ff 16427 16374 71 77 5 0 0 1974 118 118 0 0 0 0 0 0 5659 3715 1441 2116 615 1 1 3714 0 0 0 0 0 0 0 0 0 86976 20810 0
cpu5 1 0 273200 132423 134324 39356 537132479700 11555079637 2917078
domain0 22 34464 34105 215 383 135 0 0 34074 791 791 0 0 0 0 0 791 5255 4561 205 779 543 0 0 4532 0 0 0 0 0 0 0 0 0 14842 2741 0
domain1 ff 15127 15083 35 47 11 0 0 2688 126 126 0 0 0 0 0 0 4553 2928 1153 1798 570 0 0 2928 0 0 0 0 0 0 0 0 0 80126 16853 0
cpu6 0 0 285169 138295 138452 44061 517374796871 12016525074 3001982
domain0 44 38288 38170 70 111 38 0 0 38157 641 641 0 0 0 0 0 641 5832 5424 42 444 402 0 0 5424 0 0 0 0 0 0 0 0 0 13452 3378 0
domain1 ff 15901 15834 60 75 8 1 0 2023 117 117 0 0 0 0 0 0 5231 3271 1432 2254 656 0 0 3271 0 0 0 0 0 0 0 0 0 80939 18100 0
cpu7 6 0 264580 128190 144356 41145 497275125272 12070082308 2985981
domain0 88 36128 36024 49 90 40 0 0 36028 767 766 1 33 0 0 0 766 5635 5272 37 388 349 0 0 5270 0 0 0 0 0 0 0 0 0 13360 2345 0
domain1 ff 15161 15097 64 73 5 0 0 1968 144 144 0 0 0 0 0 0 5125 3303 1334 2094 593 0 0 3303 0 0 0 0 0 0 0 0 0 89851 14521 0

Domain statistics
-----------------
One of these is produced per domain for each cpu described. (Note that if
CONFIG_SMP is not defined, *no* domains are utilized and these lines
will not appear in the output.)

domain<N> <cpumask> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

The first field is a bit mask indicating what cpus this domain operates over.

The next 24 are a variety of load_balance() statistics in grouped into types
of idleness (idle, busy, and newly idle):

    1)  # of times in this domain load_balance() was called when the 
        cpu was idle
    2)  # of times in this domain load_balance() checked but found
        the load did not require balancing when the cpu was idle
    3)  # of times in this domain load_balance() tried to move one or
        more tasks and failed, when the cpu was idle
    4)  sum of imbalances discovered (if any) with each call to
        load_balance() in this domain when the cpu was idle
    5)  # of times in this domain pull_task() was called when the cpu 
        was idle
    6)  # of times in this domain pull_task() was called even though
        the target task was cache-hot when idle
    7)  # of times in this domain load_balance() was called but did 
        not find a busier queue while the cpu was idle
    8)  # of times in this domain a busier queue was found while the 
        cpu was idle but no busier group was found
    9)  # of times in this domain load_balance() was called when the 
        cpu was busy
    10) # of times in this domain load_balance() checked but found the 
        load did not require balancing when busy
    11) # of times in this domain load_balance() tried to move one or
        more tasks and failed, when the cpu was busy
    12) sum of imbalances discovered (if any) with each call to
        load_balance() in this domain when the cpu was busy
    13) # of times in this domain pull_task() was called when busy
    14) # of times in this domain pull_task() was called even though the
        target task was cache-hot when busy
    15) # of times in this domain load_balance() was called but did not
        find a busier queue while the cpu was busy
    16) # of times in this domain a busier queue was found while the cpu
        was busy but no busier group was found
    17) # of times in this domain load_balance() was called when the
        cpu was just becoming idle
    18) # of times in this domain load_balance() checked but found the
        load did not require balancing when the cpu was just becoming idle
    19) # of times in this domain load_balance() tried to move one or more
        tasks and failed, when the cpu was just becoming idle
    20) sum of imbalances discovered (if any) with each call to
        load_balance() in this domain when the cpu was just becoming idle
    21) # of times in this domain pull_task() was called when newly idle
    22) # of times in this domain pull_task() was called even though the
        target task was cache-hot when just becoming idle
    23) # of times in this domain load_balance() was called but did not
        find a busier queue while the cpu was just becoming idle
    24) # of times in this domain a busier queue was found while the cpu
        was just becoming idle but no busier group was found

   Next three are active_load_balance() statistics:

    25) # of times active_load_balance() was called
    26) # of times active_load_balance() tried to move a task and failed
    27) # of times active_load_balance() successfully moved a task

   Next three are sched_balance_exec() statistics:

    28) sbe_cnt is not used
    29) sbe_balanced is not used
    30) sbe_pushed is not used

   Next three are sched_balance_fork() statistics:

    31) sbf_cnt is not used
    32) sbf_balanced is not used
    33) sbf_pushed is not used

   Next three are try_to_wake_up() statistics:

    34) # of times in this domain try_to_wake_up() awoke a task that
        last ran on a different cpu in this domain
    35) # of times in this domain try_to_wake_up() moved a task to the
        waking cpu because it was cache-cold on its own cpu anyway
    36) # of times in this domain try_to_wake_up() started passive balancing

                                                《Documentation/scheduler/sched-stats.rst》
 

1.2.2 负载均衡的流程

1.2.2.1 流程图

                                        《Linux内核深度解析》P107

                                        《奔跑吧Linux内核》卷1:基础架构;P530

1.2.2.2 找出最忙的调度组: find_busiest_group();

相关函数:update_sd_lb_stats()、calculate_imbalance() 和 update_sg_lb_stats();

《Linux内核深度解析》P107

《深⼊理解LINUX内核》P288

《深⼊Linux内核架构》P99

《奔跑吧Linux内核》卷1:基础架构;P531

1.2.2.3 detach_tasks() / attach_tasks()

detach_tasks()

        便利最繁忙的就绪队列中的所有的进程,找出适合 被迁移的进程,然后让这些进程退出就绪队列。

attach_tasks()

        把刚才从最繁忙就绪队列中迁出的进程都迁⼊当前CPU的就绪队列中。

                                《奔跑吧Linux内核》卷1:基础架构;P530

1.2.2.4 迁移线程: migration/<cpu_id>

如果负载均衡失败,即没有迁移⼀个进程,那么为最忙处理器设置主动负载均衡标志,记录当前处理器作 为迁移⽬标,向最忙处理器的停机⼯作队列添加⼀个⼯作,⼯作函数是active_load_balance_cpu_stop,唤 醒最忙处理器的迁移线程。迁移线程将会从停机⼯作队列取出⼯作,执⾏主动的负载均衡。

                                《Linux内核深度解析》P107

                                《深⼊Linux内核架构》P100

1.2.3 进程迁移的代价

1.3 限期调度类 的 负载均衡

调度器选择下⼀个限期进程的时候,如果当前正在执⾏的进程是限 期进程,将会试图从限期进程超载的处理器把限期进程拉过来。

限期进程超载的定义:

  1. 限期运⾏队列⾄少有2个限期进程。
  2. ⾄少有⼀个限期进程绑定到多个处理器。

                                                        《Linux内核深度解析》P96

1.4 实时调度类 的 负载均衡

调度器选择下一个实时进程时,如果当前处理器的实时运⾏队列中的进程的最⾼调度优先级⽐当前正在执⾏的进程 的调度优先级低,将会试图从实时进程超载的处理器把可推送实时进程拉过来。

实时进程超载的定义:

  1. 实时运⾏队列⾄少有2个实时进程。
  2. ⾄少有⼀个可推送实时进程。 可推送实时进程是指绑定到多个处理器的实时进程,可以在处理器之间迁移。
                                                            《Linux内核深度解析》P98

1.5 调试

/sys/kernel/debug/tracing/events/sched/sched_migrate_task/

2 单个处理器核的负载(使用率)

可以通过命令“sar -P ALL 1”查看处理器核的使用率信息,也可以生成使用率图表,请看Linux下性能分析的可视化图表工具_linux 热力图-CSDN博客

3 系统负载

3.1 1分钟、5分钟、 15分钟内的平均负载

3.1.1 简介

展⽰了系统中的负载需求:系统中处于可运⾏状态的,以及不可中断等待状态的任务的数量。
                                《BPF之巅.洞悉Linux系统和应⽤性能》P198

1分钟、5分钟、15分钟的平均负载数据含义请看一篇读懂|Linux系统平均负载_系统负载怎么算-CSDN博客

3.1.2 查看方式

执行以下命令

  1.         uptime
  2.         top / htop
  3.         w
  4.         cat /proc/loadavg

3.2 Pressure Stall Information (PSI)

3.2.1 简介

An interface has now been added in Linux 4.20 that provides such a breakdown: pressure stall information (PSI), which gives averages for CPU, memory, and I/O.
                                《SystemsPerformance_ EnterpriseandtheCloud(2020,Pearson)》P257

3.2.2 /proc/pressure/cpu

# cat /proc/pressure/cpu 
some avg10=0.00 avg60=0.00 avg300=0.00 total=6305749
full avg10=0.00 avg60=0.00 avg300=0.00 total=0

The "some" line indicates the share of time in which at least some tasks are stalled on a given resource.
The "full" line indicates the share of time in which all non-idle tasks are stalled on a given resource simultaneously.
                                Documentation/accounting/psi.rst

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/diannao/2485.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

银河麒麟V10 SP1服务器客户端定时数据同步

银河麒麟V10 SP1服务器客户端定时数据同步 0.概述 当前只测试了将数据从客户端往服务端推送&#xff0c;两个客户端分别推送不同的数据 1.环境 三台电脑均为银河麒麟V10SP1桌面操作系统 服务器IP&#xff1a;192.168.1.51 用户名&#xff1a;wlh 客户端IP&#xff1a;192…

OpenStack云计算(十一)——OpenStack网络管理,验证OpenStack网络资源模型,验证来巩固和加深对OpenStack网络资源模型的理解

项目实训一 【实训题目】 验证OpenStack网络资源模型 【实训目的】 通过验证来巩固和加深对OpenStack网络资源模型的理解。 【实训准备】 &#xff08;1&#xff09;复习Neutron网络资源模型。 &#xff08;2&#xff09;重点理解网络、子网、端口和路由器的概念。 【实…

用 LM Studio 1 分钟搭建可在本地运行大型语言模型平台替代 ChatGPT

&#x1f4cc; 简介 LM Studio是一个允许用户在本地离线运行大型语言模型&#xff08;LLMs&#xff09;的平台&#xff0c;它提供了一种便捷的方式来使用和测试这些先进的机器学习模型&#xff0c;而无需依赖于互联网连接。以下是LM Studio的一些关键特性&#xff1a; 脱机&am…

【笔记django】创建一个app

创建app 错误 raise ImproperlyConfigured( django.core.exceptions.ImproperlyConfigured: Cannot import rules. Check that dvadmin.rules.apps.RulesConfig.name is correct.原因 刚创建的rules的app被手动移动到了dvadmin目录下 而dvadmin/rules/apps.py的内容还是&…

kubeadmin搭建自建k8s集群

一、安装要求 在开始之前&#xff0c;部署Kubernetes集群的虚拟机需要满足以下几个条件&#xff1a; 操作系统 CentOS7.x-86_x64硬件配置&#xff1a;2GB或更多RAM&#xff0c;2个CPU或更多CPU&#xff0c;硬盘30GB或更多【注意master需要两核】可以访问外网&#xff0c;需要…

万界星空科技电机行业MES+商业电机行业开源MES+项目合作

要得出mes系统解决方案在机电行业的应用范围&#xff0c;我们先来看一下传统机电行业的管理难题&#xff1a; 1、 产品标准化程度较低&#xff0c;制造工艺复杂&#xff0c;生产周期较长&#xff0c;产品质量不稳定&#xff1b; 2、 自动化程度低&#xff0c;大多数工序以手工…

【linux】匿名管道|进程池

1.进程为什么要通信&#xff1f; 进程也是需要某种协同的&#xff0c;所以如何协同的前提条件(通信) 通信数据的类别&#xff1a; 1.通知就绪的 2.单纯的数据 3.控制相关的信息 2.进程如何通信&#xff1f; 进程间通信&#xff0c;成本会高一点 进程间通信的前提&#xff0c;先…

2024年最新 Windows\Linux 后台运行程序注解

一、前言 有时候需要在后台运行程序&#xff0c;查了查网上现有的程序有些运行不了&#xff0c;跑通了之后简单记录一下&#xff0c;为后面的同学躺一下雷 1、Linux 系统 Linux后台运行进程时&#xff0c;通常使用如下方法&#xff1a; &#xff08;1&#xff09;程序挂后台…

算法练习(2)——约瑟夫环和坐标公式的推导

看一下上面的牛客题。题目的意思是n个小朋友围成一个圆环&#xff0c;编号从0开始&#xff0c;数m个数时&#xff0c;让小朋友出列&#xff0c;然后出列小朋友的下一个位置为0&#xff0c;继续数m个数&#xff0c;然后小朋友出来&#xff0c;直到最后一个小朋友&#xff0c;然后…

SVN小乌龟汉化问题

1.首先确认中文语言包和SVN版本需要一致&#xff08;点击右键 选择最后一个选项即可查看&#xff09; 官网链接 点击这个官网链接可以下载对应版本的中文包 2.下载好之后直接无脑下一步安装即可 3.如果还是没有中文&#xff0c;找到这个文件夹&#xff0c;把里面的内容全部删…

图解《图搜索算法》及代码实现

关注我&#xff0c;持续分享逻辑思维&管理思维&#xff1b; 可提供大厂面试辅导、及定制化求职/在职/管理/架构辅导&#xff1b; 有意找工作的同学&#xff0c;请参考博主的原创&#xff1a;《面试官心得--面试前应该如何准备》&#xff0c;《面试官心得--面试时如何进行自…

GPT与GAN结合生成图像——VQGAN原理解析

1、前言 这篇文章&#xff0c;我们讲VQ_GAN&#xff0c;这是一个将特征向量离散化的模型&#xff0c;其效果相当不错&#xff0c;搭配Transformer&#xff08;GPT&#xff09;或者CLIP使用&#xff0c;达到的效果在当时可谓是令人拍案叫绝&#xff01; 原论文&#xff1a;Tam…

【视频异常检测】Open-Vocabulary Video Anomaly Detection 论文阅读

Open-Vocabulary Video Anomaly Detection 论文阅读 AbstractMethod3.1. Overall Framework3.2. Temporal Adapter Module3.3. Semantic Knowledge Injection Module3.4. Novel Anomaly Synthesis Module3.5. Objective Functions3.5.1 Training stage without pseudo anomaly …

网盘——查看文件

本文主要讲解文件操作过程中&#xff0c;查看文件如何实现&#xff0c;实现步骤如下&#xff1a; 1、实现步骤&#xff1a; A、首先客户端发送查看请求&#xff08;包含目录信息&#xff09; B、服务器将文件名字还有文件的类型发送给客户端&#xff08;只发送文件的名字&am…

Linux2.6内核进程调度队列

目录 运行队列runqueue 活跃队列&过期队列 queue[140]&优先级&队列数组下标 bitmap[5]&O(1)调度算法 nr_active active指针和expired指针 O(1)调度算法之调度过程 本篇是Linux进程概念篇的最后一篇&#xff0c;Linux2.6内核是一个具体的/可行的/实际的存…

【Node.js】03 —— HTTP 模块探索

&#x1f31f;Node.js之HTTP模块探索✨ &#x1f31f;引言 在网络编程中&#xff0c;HTTP协议无处不在。在Node.js的世界里&#xff0c;我们可以通过内置的http模块来轻松创建HTTP服务器和客户端&#xff0c;实现数据的接收和发送。今天就让我们一起打开这扇门&#xff0c;探索…

Fisher判别示例:鸢尾花(iris)数据(R)

先读取iris数据&#xff0c;再用程序包MASS&#xff08;记得要在使用MASS前下载好该程序包&#xff09;中的线性函数lda()作判别分析&#xff1a; data(iris) #读入数据 iris #展示数据 attach(iris) #用变量名绑定对应数据 library(MASS) #加载MASS程序包 ldlda(Species~…

Python打怪升级(4)

在计算机领域常常有说"合法"和"非法"指的是:是否合理&#xff0c;是否有效&#xff0c;并不是指触犯了法律。 random.randint(begin,end) 详细讲解一下这个random是指模板&#xff0c;也就是别人写好的代码直接来用&#xff0c;在Python当中&#xff0c;…

C语言入门课程学习笔记1

C语言入门课程学习笔记1 第1课 - 概论第2课 -helloworld第3课 -数据输出第4课 -数据类型与变量第5课 - 深入数据类型与变量第6课 - 类型与变量编程练习第7课 - 程序中的数据输入 本文学习自狄泰软件学院 唐佐林老师的 C语言入门课程&#xff0c;图片全部来源于课程PPT&#xff…

食用油5G智能工厂数字孪生可视化平台,推进食品制造业数字化转型

食用油5G智能工厂数字孪生可视化平台&#xff0c;推进食品制造业数字化转型。在食用油产业中&#xff0c;数字化转型已成为提升生产效率、优化供应链管理、确保产品质量和满足消费者需求的关键。食用油5G智能工厂数字孪生可视化平台作为这一转型的重要工具&#xff0c;正在推动…