Goreplay来做流量回放

最近做项目，用到goreplay来做流量回放，什么是goreplay？

GoReplay is an open-source network monitoring tool which can record your live traffic, and use it for shadowing, load testing, monitoring and detailed analysis.

本质上就是通过监控网络，录制http请求信息（对应用无入侵），然后再把请求重新播放出来。

这是官方给出来运行原理图。

关于goreplay的使用方式，命令用法官方文档给的很全面了，不作赘述，本篇文章关注的是除杂（过滤）。官方本来是支持过滤filtering的，但过滤的范围有限，目前只有Allow url regexp,Disallow url regexp,Filter based on regexp of header,Filter based on HTTP method，也就是对url,header,method三个方向作了过滤，我的需求是对body过滤，不包括在范围内（也有可能我没找到对应文档）

官方找不到对应办法，只能“曲线救国”了，因为goreplay录制的内容是文本性质，所以可以对录制内容下手，把自己认定的杂质除掉，然后再播放，这样就达到效果了。

录制：

我是在windows下demo的，所以下载goreplay的winodwst版本【https://github.com/buger/goreplay/releases/download/v1.3.2/gor-1.3.2_windows.zip】，同时还需要安装WinPcap配合【https://www.winpcap.org/install/bin/WinPcap_4_1_3.exe】，这样就可以通过命令进行录制了：

gor --input-raw :5000 --output-file request.gor

被录制的服务是一个api，实现如下

[HttpPost("/order")]
public IActionResult Post([FromBody] Order order)
{_logger.LogInformation(System.Text.Json.JsonSerializer.Serialize(order));order.Status = 200;return new JsonResult(order);
}public class Order
{public string Code { get; set; }public decimal Amount { get; set; }public int Status { get; set; }
}

gorreplay录制的内容格式如下：

1 c9051388c0a8000437c39d7f 1629555358511143000 0
POST /pay HTTP/1.1
Content-Type: application/json
User-Agent: PostmanRuntime/7.28.3
Accept: */*
Postman-Token: 3a49aba0-2e9b-4f1d-8fc8-01fb8b15d620
Host: 192.168.0.7:5000
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 92{"code":"3a13abfe-d9cb-4e11-ba90-738e93d2cb07","amount":908.59,"status":552
}
????????????
1 c9051388c0a8000437c39d7f 1629555358813757000 0
POST /pay HTTP/1.1
Content-Type: application/json
User-Agent: PostmanRuntime/7.28.3
Accept: */*
Postman-Token: 3a49aba0-2e9b-4f1d-8fc8-01fb8b15d620
Host: 192.168.0.7:5000
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 92{"code":"3a13abfe-d9cb-4e11-ba90-738e93d2cb07","amount":908.59,"status":552
}
????????????

除杂

文中的json就是我请求的body，就要依据这个json中的一些数据来除杂，所以就得写段代码解决这个事情。

其实原理很简单，因为我知道我请求的都是json，所以就是从录制的文件中，按段切割，很明细就是三只猴子????????????为分割点，把内容中的json查询出来，然后运用上规则引擎达到过滤作用，把符合条件的数据留下，不符合条件的数据就除杂，最后保存成一个新文件，供流量回放时使用。

规则引擎
桂素伟，公众号：桂迹一个简单的规则引擎例子

步骤确定了，下面是代码：

using RulesEngine;
using RulesEngine.Models;
using RulesEngine.Extensions;
using System.IO;
using System.Text;
using System.Text.RegularExpressions;
using Newtonsoft.Json.Converters;
using Newtonsoft.Json;
using System.Dynamic;var path = @"C:\MyFile\Source\Repos\Asp.NetCoreExperiment\Asp.NetCoreExperiment\GoReplay\GoReplayDemo01\request_0.gor";
var expression = "input1.amount >= 900.00";
await ImpurityRemoval(path, expression);/// <summary>
/// 除杂方法，会重新生成一个带有日期时间的新.gor文件
/// </summary>
static async Task ImpurityRemoval(string path, string expression)
{using var readFile = new StreamReader(path, Encoding.UTF8);using var writeFile = new StreamWriter(@$"{path.Replace(Path.GetFileName(path), Path.GetFileNameWithoutExtension(path) + "_" + DateTime.Now.ToString("yyyyMMddHHmmss") + Path.GetExtension(path))}", true, Encoding.UTF8);var split = "????????????";string? line;var request = new StringBuilder();while ((line = await readFile.ReadLineAsync()) != null){if (line != split){request.Append(line + "\n");}else{request.Append(line + "\n");var list = GetJson(request.ToString());foreach (var item in list){var converter = new ExpandoObjectConverter();var entity = JsonConvert.DeserializeObject<ExpandoObject>(item, converter);if (await Filter(entity, expression)){await writeFile.WriteAsync(request.ToString());}}request.Clear();}}
}
/// <summary>
/// 获取json，这里没有完全测试
/// </summary>
static List<string> GetJson(string jsonString)
{var pattern = @"\{(.|\s)*\}";var list = new List<string>();var matches = Regex.Matches(jsonString, pattern, RegexOptions.IgnoreCase);foreach (Match m in matches){list.Add(m.Value);}return list;
}
/// <summary>
/// 用规则引擎匹配过滤规则
/// </summary>
static async Task<bool> Filter(dynamic? entity, string expression)
{var workRules = new WorkflowRules();workRules.WorkflowName = "ImpurityRemoval";workRules.Rules = new List<Rule>{new Rule{RuleName="ImpurityRemoval01",SuccessEvent= "10",RuleExpressionType= RuleExpressionType.LambdaExpression,Expression= expression,          }};var rulesEngine = new RulesEngine.RulesEngine(new WorkflowRules[] { workRules });List<RuleResultTree> resultList = await rulesEngine.ExecuteAllRulesAsync("ImpurityRemoval", entity);var result = false;resultList.OnSuccess((eventName) =>{result = true;});return result;
}

回放