rsyslog管理服务日志

tech2023-11-23  92

rsyslog管理服务日志

什么和为什么:云中的日志管理 (What & Why: Log Management in Cloud)

Logging is often not the most exciting part of an application, but definitely an important one. Logging and managing logs becomes even more important in a multi node clustered environments where logs are spread across nodes. It is not a very productive proposition to log on to each node to check what is happening on a node or to trace the transaction which might be spread across multiple nodes. Moreover ability to filter and parse logs on a node is limited by native tools such as grep on *nix systems. Is there a better solution? Absolutely! Welcome to log management or centralized logging, demonstrated at a high level in following image

日志记录通常不是应用程序中最令人兴奋的部分,但绝对是重要的部分。 在多节点集群环境中,日志分散在节点之间,日志记录和管理日志变得更加重要。 登录到每个节点以检查某个节点上正在发生的事情或跟踪可能分布在多个节点上的事务,这不是一个非常有效的提议。 而且,本机工具(例如* nix系统上的grep)限制了在节点上过滤和解析日志的能力。 有更好的解决方案吗? 绝对! 欢迎使用日志管理或集中式日志记录,如下图所示

This might sound complicated to setup, but let’s look at the benefits and then see how easy it is to setup log management.

设置起来听起来很复杂,但是让我们看一下好处,然后看看设置日志管理有多么容易。

生产故障排除和应用程序调试 (Production Troubleshooting & Application Debugging)

If you are troubleshooting a live application, logs will be the lifeline which will save your day. On development environments, there are enough tools at hand which make debugging application much easier. But for an application which is live in production, logs provide key information about what is going on in the application. On the other hand too verbose logging might hamper performance of an application,but that is a separate discussion and there are tools available to control level of logging based on environments.

如果您要对正在运行的应用程序进行故障排除,日志将是生命线,这将节省您的时间。 在开发环境上,有足够的工具可以使调试应用程序变得更加容易。 但是对于正在生产中的应用程序,日志提供了有关应用程序中正在发生的事情的关键信息。 另一方面,过于冗长的日志记录可能会影响应用程序的性能,但这是一个单独的讨论,并且有一些工具可用于根据环境控制日志记录的级别。

“debugging statements stay with the program; debugging sessions are transient”

“调试语句与程序一起存在; 调试会话是暂时的”

If you have applications deployed in private/public cloud, or even deployed within enterprise in a clustered fashion, it will be much easier to troubleshoot and debug application if all your logs can be searched and parsed all in one place. Inefficiencies of scrolling over thousands of lines of raw data through multiple nodes and trying to connect dots would be solved by a log management system which provides a unified view of all logs and a much richer query interface.

如果您将应用程序部署在私有/公共云中,或者甚至以集群方式部署在企业中,那么如果可以一次搜索并解析所有日志,则对应用程序进行故障排除和调试将更加容易。 通过日志管理系统可以解决在多个节点上滚动数千行原始数据并尝试连接点的低效率问题,该系统提供了所有日志的统一视图和更丰富的查询界面。

数据分析 (Data Analysis)

A running application has multiple user interactions and transactions running and being able to gather real time insights from these would be extremely valuable.  Applications can be designed to log desired information about transactions and this information from logs can be analyzed and monitored in real time. Just imagine if CEO of a retail company could see in real time statistics of how many shopping carts are converted in actual buys. While analytics is a huge field in itself, you got the idea of possibilities.

一个正在运行的应用程序具有多个用户交互和正在运行的事务,并且能够从这些交互中收集实时见解将非常有价值。 可以将应用程序设计为记录所需的有关事务的信息,并且可以实时分析和监视来自日志的信息。 试想一下,一家零售公司的首席执行官是否可以实时查看在实际购买中转换了多少购物车的统计信息。 尽管分析本身就是一个巨大的领域,但您却想到了可能性。

监控和警报 (Monitoring and alerting)

Being able to report on errors, exceptions and crashes in real time can lead to actions which reduce downtime and much proactive response to issues in a live application. Alerts can be setup based on severity level and actions can be taken accordingly.

能够实时报告错误,异常和崩溃可以导致减少停机时间并积极响应实时应用程序中的问题的操作。 可以根据严重性级别设置警报,并可以采取相应的措施。

日志管理系统的功能 (Features of a Log management system)

While every product and service offers different flavours of features, we consolidate here some features you should look out for.

尽管每种产品和服务都提供不同的功能,但是我们在这里合并了您应该注意的一些功能。

挖掘能力 (Drill Down Capability)

Time based drill down is a very standard feature of any log management system and offers ability to look at patterns plotted vs time, which can be drilled down further. For example look at a screenshot of LogStash:

基于时间的向下钻取是任何日志管理系统的一个非常标准的功能,并且可以查看随时间绘制的模式,可以进一步向下钻取。 例如,看一下LogStash的屏幕截图:

正则表达式和过滤 (Regular expressions and filtering)

Being able to filter on random text, or being able to click on strings from the result shown on UI makes navigating and finding relevant logs much easier and quicker. Also a regular expression library like grok for example shipped with LogStash helps using regular expressions a easier job.

能够过滤随机文本,或者能够从UI上显示的结果中单击字符串,使导航和查找相关日志变得更加轻松快捷。 LogStash附带的正则表达式库(例如grok)也有助于使用正则表达式更轻松。

警报和通知 (Alerts and notifications)

A good log management system should be able to hook to multiple systems like email, websockets for delivering information. That increases reach of the system and it’s usefulness.

一个好的日志管理系统应该能够连接到多个系统,例如电子邮件,网络套接字来传递信息。 这增加了系统的覆盖面,并且很有用。

API (API)

Last but not the least, having a API of the log management system exposed can greatly increase adoption and extensibility of platform in some cases.

最后但并非最不重要的一点是,在某些情况下,公开日志管理系统的API可以大大提高平台的采用率和可扩展性。

要考虑的因素 (Factors to Consider)

Here is a definitive questionnaire you should analyze for choosing your log management system:

这是一份确定的问卷,您应该在选择您的日志管理系统时进行分析:

Which type of applications you have and do you have sufficient hooks available to plug into log management system? For example if you would like Android apps to log data to log management server, do you have a lightweight library supporting the same?

您拥有哪种类型的应用程序,您是否有足够的钩子可用于插入日志管理系统? 例如,如果您希望Android应用程序将数据记录到日志管理服务器,那么您是否有支持该应用程序的轻量级库? Does the language in which applications are developed has easier ways to communicate to a log management server? For example log4j has SysLogAppender which can talk to Syslog which in turn can talk to log management system.

开发应用程序所使用的语言是否具有与日志管理服务器进行通信的简便方法? 例如,log4j具有可以与Syslog对话的SysLogAppender,后者又可以与日志管理系统对话。 Would you like to setup your own log management systems or you would like to use log management provider’s services? (Refer to following sections for both approaches)

您要设置自己的日志管理系统还是要使用日志管理提供程序的服务? (有关两种方法,请参阅以下各节) If your application is hosted on a PaaS platform, can you integrate the platform with your own or hosted log management servers?

如果您的应用程序托管在PaaS平台上,那么您可以将该平台与自己或托管的日志管理服务器集成吗?

日志管理:自行设置 (Log Management: Set Up Yourself)

If you are planning to set up your own log management system, either within your enterprise or on hosted environments you have quite a few choices. If your organization has budgets to buy support and want enterprise grade system without a team to maintain the system inhouse, then Splunk is probably a good fit. If you have a team inhouse which is ready to look after the system and prefers a DIY approach, then you can choose from a set of tools for each specific purpose. Log storage and indexing can be done in MongoDB, ElasticSearch to name a few. Tools like LogStash, Kibana, Graphite, Greylog2 offer different features and you can mix and match to suit your needs.

如果您打算在企业内部或托管环境中设置自己的日志管理系统,则有很多选择。 如果您的组织有预算来购买支持,并且想要没有团队维护企业内部的企业级系统,那么Splunk可能是一个不错的选择。 如果您有一个内部团队准备好照顾系统并喜欢使用DIY方法,则可以针对每个特定目的从一组工具中进行选择。 日志存储和索引可以在MongoDB中完成,ElasticSearch仅举几例。 LogStash , Kibana , Graphite , Greylog2之类的工具提供了不同的功能,您可以混合搭配以适应您的需求。

日志管理即服务 (Log Management as a Service)

If you want to add Log Management as a service to your existing apps in cloud, you have choices like Loggly, papertrail, logentries among the main players. While we won’t be discussing them in detail in this post, each one offers it’s unique flavors of log management strategy.

如果要将Log Management作为服务添加到云中现有的应用程序,则可以在主要参与者之间选择Loggly , papertrail , logentries 。 尽管我们不会在本文中详细讨论它们,但每个人都提供了其独特的日志管理策略风格。

结论 (Conclusion)

You can decide not to give enough importance to logging in your applications but the benefits of logging and a good log management system are far reaching. In a world where horizontally scaled environments are the norm and cloud is being adopted day by day, a good log management system is a necessity but can prove to be a differentiator.  Organizations have already started reaping benefits of log management system from production debugging to analytics. Has your company started the journey yet? Let us know your thoughts.

您可以决定不对应用程序的登录给予足够的重视,但是日志记录和良好的日志管理系统的好处是深远的。 在一个水平扩展的环境已成为常态并且每天都在使用云的世界中,一个好的日志管理系统是必要的,但可以证明是一个与众不同的地方。 组织已经开始从生产调试到分析从日志管理系统中受益。 贵公司是否已开始旅程? 让我们知道您的想法。

翻译自: https://www.sitepoint.com/log-management-as-a-service/

rsyslog管理服务日志

相关资源:Linux 通过rsyslog配置日志服务器
最新回复(0)