影响系统性能的20个瓶颈

精选转载

lyuanping 2016-03-18 09:08:22 博主文章分类：Linux--性能调优

文章标签 性能瓶颈 文章分类 运维

影响系统性能的20个瓶颈

Zen And The Art Of Scaling - A Koan And Epigram Approach 一文中 , Russell Sullivan 提出一个很有趣的设想：一共有20种经典的瓶颈。这听起来就像只有20种基本的故事情节（20 basic story plots）那样让人怀疑。不过基于每个人不同的分类方式，这个说法或许是对的，但是在现实中，众所周知，瓶颈是无穷无尽的而且涉及方方面面。

· 数据库:

· 工作中数据大小超过可用内存 RAM

· 长短查询混合

· 写-写冲突

· 大的联合查询占光内存

· 虚拟化:

· 共享 HDD 存储，磁盘寻道挂起

· 云平台中的网络 I/O 波动

编程:

· 线程：死锁、相对于事件驱动来说过于重量级、调试、线程数与性能比非线性

· 事件驱动编程：回调的复杂性、函数调用中如何保存状态（how-to-store-state-in-function-calls）

· 缺少profile工具、缺少trace工具、缺少日志工具

· 单点故障、横向不可扩展

· 有状态的应用

· 搓设计：一台机器上能跑，几个用户也能跑，几个月后，几年后，尼玛，发现扛不住了，整个架构需要重写。

· 算法复杂度

· 依赖于诸如DNS查找等比较搞人的外部组件

· 栈空间

· 磁盘：

· 本地磁盘存取

· 随机磁盘读写 -> 磁盘寻道

· 磁盘碎片化

· 写入超过SSD容量的数据导致SSD硬盘性能降低

· 操作系统：

· 内核缓冲刷入磁盘，填充linux缓冲区缓存

· TCP缓冲区过小

· 文件描述符限制

· 功率分配

缓存:

· 不使用memcached

· HTTP中，header，etags，不压缩（headers, etags, not gzipping）

· 没有充分使用浏览器缓存功能

· 字节码缓存（如PHP）

· L1/L2缓存. 这是个很大的瓶颈. 把频繁使用的数据保持在L1/L2中. 设计到的方面很多：网络数据压缩后再发送，基于列压缩的DB中不解压直接计算等等。有TLB友好的算法。最重要的是牢固掌握以下基础知识：多核CPU、L1/L2，共享L3，NUMA内存，CPU、内存之间的数据传输带宽延迟，磁盘页缓存，脏页，TCP从CPU到DRAM到网卡的流程。

CPU:

· CPU 过载

· 上下文切换 -> 一个内核上跑了太多的线程，linux调度对于应用来说很不友好, 太多的系统调用,等等...

· IO 等待 -> 所有的CPU都挂起等待比较慢的IO

· CPU 缓存: 缓存数据是一个为了平衡不同实例有不同的值和繁重的同步缓存数据保持一致，而精心设计的一个进程。

· 背板吞吐量

网络:

· 网卡的最大输出带宽，IRQ达到饱和状态，软件中断占用了100%的CPU

· DNS查找

· 丢包

· 网络路由瞎指挥

· 网络磁盘访问

· 共享SAN（Storage Area Network)

· 服务器失败 -> 服务器无响应

过程:

· 测试时间 Testing time

· 开发时间 Development time

· 团队人数 Team size

· 预算 Budget

· 代码缺陷 Code debt

内存:

· 内存溢出 -> 杀进程，进入 swap ，越来越慢

· 内存溢出导致磁盘频繁读写（swap相关）

· 内存库开销

· 内存碎片

· Java 需要垃圾收集导致程序暂停

· C 语言的 malloc 无法分配

IDEAS, PLOTS & USING THE PREMISE SHEETS

After you come up with your own system for generating ideas, the next step is to put them in some recognizable story form (the basic plot idea), build your central conflict (the story premise sheet), then build your character and underlying themes (the thematic premise sheet.

#1 QUEST - the plot involves the Protagonist's search for a person, place or thing, tangible or intangible (but must be quantifiable, so think of this as a noun; i.e., immortality).

#2 ADVENTURE - this plot involves the Protagonist going in search of their fortune, and since fortune is never found at home, the Protagonist goes to search for it somewhere over the rainbow.

#3 PURSUIT - this plot literally involves hide-and-seek, one person chasing another.

#4 RESCUE - this plot involves the Protagonist searching for someone or something, usually consisting of three main characters - the Protagonist, the Victim & the Antagonist.

#5 ESCAPE - plot involves a Protagonist confined against their will who wants to escape (does not include some one trying to escape their personal demons).

#6 REVENGE - retaliation by Protagonist or Antagonist against the other for real or imagined injury.

#7 THE RIDDLE - plot involves the Protagonist's search for clues to find the hidden meaning of something in question that is deliberately enigmatic or ambiguous.

#8 RIVALRY - plot involves Protagonist competing for same object or goal as another person (their rival).

#9 UNDERDOG - plot involves a Protagonist competing for an object or goal that is at a great disadvantage and is faced with overwhelming odds.

#10 TEMPTATION - plot involves a Protagonist that for one reason or another is induced or persuaded to do something that is unwise, wrong or immoral.

#11 METAMORPHOSIS - this plot involves the physical characteristics of the Protagonist actually changing from one form to another (reflecting their inner psychological identity).

#12 TRANSFORMATION - plot involves the process of change in the Protagonist as they journey through a stage of life that moves them from one significant character state to another.

#13 MATURATION - plot involves the Protagonist facing a problem that is part of growing up, and from dealing with it, emerging into a state of adulthood (going from innocence to experience).

#14 LOVE - plot involves the Protagonist overcoming the obstacles to love that keeps them from consummating (engaging in) true love.

#15 FORBIDDEN LOVE - plot involves Protagonist(s) overcoming obstacles created by social mores and taboos to consummate their relationship (and sometimes finding it at too high a price to live with).

#16 SACRIFICE - plot involves the Protagonist taking action(s) that is motivated by a higher purpose (concept) such as love, honor, charity or for the sake of humanity.

#17 DISCOVERY - plot that is the most character-centered of all, involves the Protagonist having to overcome an upheavel(s) in their life, and thereby discovering something important (and buried) within them a better understanding of life (i.e., better appreciation of their life, a clearer purpose in their life, etc.)

#18 WRETCHED EXCESS - plot involves a Protagonist who, either by choice or by accident, pushes the limits of acceptable behavior to the extreme and is forced to deal with the consequences (generally deals with the psychological decline of the character).

#19 ASCENSION - rags-to-riches plot deals with the rise (success) of Protagonist due to a dominating character trait that helps them to succeed.

#20 DECISION - riches-to-rags plot deals with the fall (destruction) of Protagonist due to dominating character trait that eventually destroys their success.

(Note: Sometimes #19 & #20 are combined into rags-to-riches-to-rags (or vice versa) of a Protagonist who does (or doesn't) learn to deal with their dominating character trait.)

For an in-depth look at these plots, read the excellent "20 Master Plots and How To Build Them" by Ronald B. Tobias available through the TSA Writers Store.