Elasticsearch：Jira 连接器教程第一部分

作者：来自 Elastic Gustavo Llermaly

将我们的 Jira 内容索引到 Elaasticsearch 中以创建统一的数据源并使用文档级别安全性进行搜索。

在本文中，我们将回顾 Elastic Jira 原生连接器的一个用例。我们将使用一个模拟项目，其中一家银行正在开发一款汇款应用，需要将 Jira 中的信息集成到 Elastic 中。

原生连接器允许我们从票据、任务和其他文档中获取 Elastic 集群信息，集中数据并启用高级搜索功能。

使用此连接器的主要好处是：

Jira 中的数据与 Elasticsearch 同步。
访问高级搜索功能。
文档级安全性 (DLS) 匹配源安全性。你只能搜索 Jira 中允许你查看的内容。

步骤

配置 Jira 连接器
将文档索引到 Elasticsearch
查询数据
文档级安全性 (document level security - DLS)

配置 Jira 连接器

首先，你需要从 Jira 获取 API token 以连接到 Elasticsearch。转到此链接了解如何创建它。

将其命名为 “elastic-connector”。它应该如下所示：

获取 token 并进入 Kibana 仪表板。然后，转到原生连接器并选择 New Jira Cloud connector。

https://<YOUR_KIBANA_URL>/app/enterprise_search/content/connectors/new_connector?service_type=jira&connector_type=native

用 Kibana 端点替换 YOUR_KIBANA_URL。

将连接器命名为 “bank”，然后点击 “Create and attach an index named bank”，即可创建一个同名的新索引。

完成了！现在我们需要配置我们的 Jira 数据。

由于我们不会使用自己的 SSL 证书，因此我们将保持 “Enable SSL” 关闭。

你可以在官方文档中查看每个字段的详细信息。

激活文档级安全性 (DLS)，以便你获得有权查看文档的用户和组。

正确配置连接器后，你可以继续同步数据，如下所示。从 Jira 获取数据可能需要几分钟。

完整内容（Full Content）：索引所有 Jira 文档。
增量内容（Incremental Content）：仅索引自上次完整内容同步以来的更改。
访问控制（Access Control）：在安全索引中索引 Jira 用户以激活 DLS。

我们可以检查连接器的概述来查看同步是否成功。

在 “Documents” 选项卡中，我们可以准确地看到使用连接器获取的数据。第一次同步的对象是：

Projects
Issues
Attachments

将文档索引到 Elasticsearch

我们不仅限于跨连接器文档进行搜索。Elasticsearch 允许你使用单个查询搜索多个索引。

在我们的示例中，我们将其他文档索引到 galactic_documents 索引中，以了解搜索如何与多个数据源配合使用：

GBFF 合规手册
Galactic Banking App 用户指南
技术规格报告

但在索引之前，我们将为每个字段创建优化映射：

PUT /galactic_documents
{"mappings": {"properties": {"document_id": {"type": "keyword"},"title": {"type": "text","fields": {"raw": {"type": "keyword"}}},"content": {"type": "text"},"release_date": {"type": "date","format": "yyyy-MM-dd"},"page_count": {"type": "integer"},"tags": {"type": "keyword"}}}
}

配置映射后，我们现在可以索引：

POST galactic_documents/_bulk
{ "index": { "_index": "galactic_documents", "_id": "1" } }
{ "document_id": "GBFF-001", "title": "Compliance Manual of the GBFF", "content": "This document sets forth the compliance standards for intergalactic financial entities: Quantum-level data encryption to guarantee security in all transactions. Mandatory multi-factor authentication for all users and administrators. Quarterly reviews of security policies and access audit logs.", "release_date": "2024-01-01", "page_count": 5, "tags": ["compliance", "security"] }
{ "index": { "_index": "galactic_documents", "_id": "2" } }
{ "document_id": "GBFF-002", "title": "User Guide for the Galactic Banking App", "content": "Welcome to the Galactic Banking application by Interstellar Finance Corp. Here you can: Transfer galactic credits to any registered account across the Milky Way. Check your balance and manage your investments in real-time. Access interplanetary loans with ease. For your security, use multi-factor authentication each time you log in.", "release_date": "2024-01-01", "page_count": 3, "tags": ["user guide", "application"] }
{ "index": { "_index": "galactic_documents", "_id": "3" } }
{ "document_id": "GBFF-003", "title": "Technical Specifications Report - Galactic Banking Project", "content": "This report details the technical architecture of the Galactic Banking application: Microservices-based backend for scalability and performance. Secure communication protocols utilizing quantum encryption. Transaction management adapted to environments with gravity variations and time dilation.", "release_date": "2024-01-01", "page_count": 7, "tags": ["technical", "specifications", "architecture"] }

查询数据

现在我们有了 Jira 对象和文档，我们可以一起搜索它们。

GET bank,galactic_documents/_search
{"query": {"multi_match": {"query": "galactic moon","fields": ["content","title","*description","*summary"]}}
}

查询 “galactic moon” 将为我们获取 Jira 对象和我们索引的文档：

{"took": 3,"timed_out": false,"_shards": {"total": 3,"successful": 3,"skipped": 0,"failed": 0},"hits": {"total": {"value": 3,"relation": "eq"},"max_score": 1.2613049,"hits": [{"_index": "bank","_id": "Marketing Mars-MM-2","_score": 1.2613049,"_source": {"Type": "Task","Custom_Fields": {"Satisfaction": null,"Approvals": null,"Change reason": null,"Epic Link": null,"Actual end": null,"Design": null,"Campaign assets": null,"Department": null,"Story point estimate": null,"Approver groups": null,"[CHART] Date of First Response": null,"Request Type": null,"Campaign goals": null,"Project overview key": null,"Related projects": null,"Campaign type": null,"Impact": null,"Request participants": [],"Locked forms": null,"Time to first response": null,"Work category": null,"Audience": null,"Open forms": null,"Details": null,"Sprint": null,"Stakeholders": null,"Marketing asset type": null,"Submitted forms": null,"Start date": null,"Actual start": null,"Category": null,"Change risk": null,"Target start": null,"Issue color": null,"Parent Link": {"hasEpicLinkFieldDependency": false,"showField": false,"nonEditableReason": {"reason": "EPIC_LINK_SHOULD_BE_USED","message": "To set an epic as the parent, use the epic link instead"}},"Format": null,"Target end": null,"Approvers": null,"Team": null,"Change type": null,"Satisfaction date": null,"Request language": null,"Amount": null,"Rank": "0|i0003j:","Affected services": null,"Type": null,"Time to resolution": null,"Total forms": null,"[CHART] Time in Status": null,"Organizations": [],"Flagged": null,"Project overview status": null},"Issue": {"statuscategorychangedate": "2024-11-01T17:52:30.550-0300","issuetype": {"avatarId": 10318,"hierarchyLevel": 0,"name": "Task","self": "https://xxxx.atlassian.net/rest/api/2/issuetype/10017","description": "A small, distinct piece of work.","entityId": "f30ea676-7b3d-44ad-9858-558081742a2e","id": "10017","iconUrl": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10318?size=medium","subtask": false},"components": [],"timespent": null,"timeoriginalestimate": null,"project": {"simplified": true,"avatarUrls": {"48x48": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10418","24x24": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10418?size=small","16x16": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10418?size=xsmall","32x32": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10418?size=medium"},"name": "Marketing Mars","self": "https://xxxx.atlassian.net/rest/api/2/project/10003","id": "10003","projectTypeKey": "business","key": "MM"},"description": null,"fixVersions": [],"aggregatetimespent": null,"resolution": null,"timetracking": {},"security": null,"aggregatetimeestimate": null,"attachment": [],"resolutiondate": null,"workratio": -1,"summary": "Conquer the moon","issuerestriction": {"issuerestrictions": {},"shouldDisplay": true},"watches": {"self": "https://xxxx.atlassian.net/rest/api/2/issue/MM-2/watchers","isWatching": true,"watchCount": 1},"lastViewed": "2024-11-01T17:52:34.925-0300","creator": {"accountId": "712020:88983800-6c97-469a-9451-79c2dd3732b5","emailAddress": "contornan_cliche.0y@icloud.com","avatarUrls": {"48x48": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png","24x24": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png","16x16": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png","32x32": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png"},"displayName": "Tomas Murua","accountType": "atlassian","self": "https://xxxx.atlassian.net/rest/api/2/user?accountId=712020%3A88983800-6c97-469a-9451-79c2dd3732b5","active": true,"timeZone": "Chile/Continental"},"subtasks": [],"created": "2024-11-01T17:52:30.289-0300","reporter": {"accountId": "712020:88983800-6c97-469a-9451-79c2dd3732b5","emailAddress": "contornan_cliche.0y@icloud.com","avatarUrls": {"48x48": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png","24x24": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png","16x16": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png","32x32": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png"},"displayName": "Tomas Murua","accountType": "atlassian","self": "https://xxxx.atlassian.net/rest/api/2/user?accountId=712020%3A88983800-6c97-469a-9451-79c2dd3732b5","active": true,"timeZone": "Chile/Continental"},"aggregateprogress": {"total": 0,"progress": 0},"priority": {"name": "Medium","self": "https://xxxx.atlassian.net/rest/api/2/priority/3","iconUrl": "https://xxxx.atlassian.net/images/icons/priorities/medium.svg","id": "3"},"labels": [],"environment": null,"timeestimate": null,"aggregatetimeoriginalestimate": null,"versions": [],"duedate": null,"progress": {"total": 0,"progress": 0},"issuelinks": [],"votes": {"hasVoted": false,"self": "https://xxxx.atlassian.net/rest/api/2/issue/MM-2/votes","votes": 0},"comment": {"total": 0,"comments": [],"maxResults": 0,"self": "https://xxxx.atlassian.net/rest/api/2/issue/10018/comment","startAt": 0},"assignee": null,"worklog": {"total": 0,"maxResults": 20,"startAt": 0,"worklogs": []},"updated": "2024-11-01T17:52:42.711-0300","status": {"name": "To Do","self": "https://xxxx.atlassian.net/rest/api/2/status/10014","description": "","iconUrl": "https://xxxx.atlassian.net/","id": "10014","statusCategory": {"colorName": "blue-gray","name": "To Do","self": "https://xxxx.atlassian.net/rest/api/2/statuscategory/2","id": 2,"key": "new"}}},"id": "Marketing Mars-MM-2","_timestamp": "2024-11-01T17:52:42.711-0300","Key": "MM-2","_allow_access_control": ["account_id:712020:88983800-6c97-469a-9451-79c2dd3732b5","name:Tomas-Murua"]}},{"_index": "galactic_documents","_id": "2","_score": 0.61183906,"_source": {"document_id": "GBFF-002","title": "User Guide for the Galactic Banking App","content": "Welcome to the Galactic Banking application by Interstellar Finance Corp. Here you can: Transfer galactic credits to any registered account across the Milky Way. Check your balance and manage your investments in real-time. Access interplanetary loans with ease. For your security, use multi-factor authentication each time you log in.","release_date": "2024-01-01","page_count": 3,"tags": ["user guide","application"]}},{"_index": "galactic_documents","_id": "3","_score": 0.5029222,"_source": {"document_id": "GBFF-003","title": "Technical Specifications Report - Galactic Banking Project","content": "This report details the technical architecture of the Galactic Banking application: Microservices-based backend for scalability and performance. Secure communication protocols utilizing quantum encryption. Transaction management adapted to environments with gravity variations and time dilation.","release_date": "2024-01-01","page_count": 7,"tags": ["technical","specifications","architecture"]}}]}
}

如果文档太长，你可以将选项 _source 添加到查询中，以仅包含你需要的字段。如果你只想删除一些字段，我们将在本系列的第二部分介绍该选项。

文档级安全性 (DLS)

我们现在将配置文档级安全性 (DLS)，以将 Jira 权限与 Elasticsearch 中的权限相匹配，以便用户搜索时只能看到他们在 Jira 中被允许看到的内容。

首先，我们将转到 Elastic Cloud 中连接器的控制面板，然后单击 Access Control Sync。

此同步将带来 Jira 用户的访问和权限信息。为了测试这一点，我创建了另一个 Jira 板（board），但用户 “Gustavo” 无权访问该板。

注意：创建板后，不要忘记运行内容同步。你可以运行一次性同步，也可以按计划运行。

让我们开始检查新板中的文档是否存在：

GET bank/_search
{"_source": ["Issue.summary"],"query": {"match": {"Issue.project.name": "Marketing Mars"}}
}

我们可以有效地看到以下问题：

{"took": 2,"timed_out": false,"_shards": {"total": 2,"successful": 2,"skipped": 0,"failed": 0},"hits": {"total": {"value": 3,"relation": "eq"},"max_score": 0.7473189,"hits": [{"_index": "bank","_id": "Marketing Mars-MM-1","_score": 0.7473189,"_source": {"Issue": {"summary": "Conquer Mars"}}},{"_index": "bank","_id": "Marketing Mars-MM-3","_score": 0.7473189,"_source": {"Issue": {"summary": "Conquering Earth"}}},{"_index": "bank","_id": "Marketing Mars-MM-2","_score": 0.7473189,"_source": {"Issue": {"summary": "Conquer the moon"}}}]}
}

但是，由于用户 “Gustavo” 没有访问权限，因此他应该无法看到它们。

让我们在 ACL 过滤器索引中查找用户的文档以查看他们的权限。

GET .search-acl-filter-bank/_search

响应：

{"_index": ".search-acl-filter-bank","_id": "63c04b092341bff4fff6e0cb","_score": 1,"_source": {"created_at": "2024-11-01T23:19:35.784996+00:00","id": "63c04b092341bff4fff6e0cb","_timestamp": "2024-11-01T05:42:04.410478+00:00","identity": {"account_id": "account_id:63c04b092341bff4fff6e0cb","email_address": null,"display_name": "name:Gustavo","locale": "locale:en_US"},"query": {"template": {"source": """{"bool": {"should": [{"bool": {"must_not": {"exists": {"field": "_allow_access_control"}}}},{"terms": {"_allow_access_control.enum": {{#toJson}}access_control{{/toJson}}}}]}}""","params": {"access_control": ["account_id:63c04b092341bff4fff6e0cb","group_id:d3f28403-7e99-4262-8f11-77a75bcd33d8","role_key:jira-software"]}}}}
}

该索引包括用户 ID 及其所有 Jira 组。我们需要将用户访问控制中的内容与每个文档中的字段 _allowed_access_control 进行匹配。

我们将使用以下命令为 Gustavo 创建 API 密钥。你必须从上一步复制 query.template 值：

POST /_security/api_key
{"name": "gustavo","expiration": "30d","role_descriptors": {"jira-role": {"index": [{"names": ["bank","galactic_documents"],"privileges": ["read","view_index_metadata"],"query": {"template": {"params": {"access_control": ["account_id:63c04b092341bff4fff6e0cb","group_id:d3f28403-7e99-4262-8f11-77a75bcd33d8","role_key:jira-software"]},"source": """{"bool": {"should": [{"bool": {"must_not": {"exists": {"field": "_allow_access_control"}}}},{"terms": {"_allow_access_control.enum": {{#toJson}}access_control{{/toJson}}}}]}}"""}}}]}}
}

请注意，我们仅通过此选项授予对本文中索引的访问权限。

为 Gustavo 创建 API 密钥的响应如下：

{"id": "yLa1FJMBU4bZPaw5Stnl","name": "gustavo","expiration": 1733811245816,"api_key": "UrGdsnDFSyGxjQvLayw5jQ","encoded": "eUxhMUZKTUJVNGJaUGF3NVN0bmw6VXJHZHNuREZTeUd4alF2TGF5dzVqUQ=="
}

你可以使用 curl 来测试我们是否可以使用 API KEY 运行搜索，并且它不会从 Marketing board 获取信息，因为 Gustavo 无权访问它。

curl --location --request GET 'https://interstellar-finance-corp.es.us-central1.gcp.cloud.es.io/bank/_search' \
--header 'Authorization: ApiKey eUxhMUZKTUJVNGJaUGF3NVN0bmw6VXJHZHNuREZTeUd4alF2TGF5dzVqUQ==' \
--header 'Content-Type: application/json' \
--data '{"_source": ["Issue.summary"],"query": {"match": {"Issue.project.name": "Marketing Mars"}}
}'

响应：

{"took": 0,"timed_out": false,"_shards": {"total": 1,"successful": 1,"skipped": 0,"failed": 0},"hits": {"total": {"value": 0,"relation": "eq"},"max_score": null,"hits": []}
}

我们可以看到，Gustavo 没有获得任何信息，因为他没有访问权限。现在，让我们用他被允许查看的 board 文件进行测试：

curl --location --request GET 'https://interstellar-finance-corp.es.us-central1.gcp.cloud.es.io/bank/_search?pretty=true' \
--header 'Authorization: ApiKey eUxhMUZKTUJVNGJaUGF3NVN0bmw6VXJHZHNuREZTeUd4alF2TGF5dzVqUQ==' \
--header 'Content-Type: application/json' \
--data '{"_source": ["Issue.summary"],"query": {"match": {"Issue.project.name": "Galactic Banking Project"}}
}'

响应：

{"took" : 7,"timed_out" : false,"_shards" : {"total" : 2,"successful" : 2,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 3,"relation" : "eq"},"max_score" : 3.1784885,"hits" : [{"_index" : "bank","_id" : "Galactic Banking Project-GBP-3","_score" : 3.1784885,"_source" : {"Issue" : {"summary" : "Intergalactic Security and Compliance"}}},{"_index" : "bank","_id" : "Galactic Banking Project-GBP-2","_score" : 0.5469647,"_source" : {"Issue" : {"summary" : "Bank Application Frontend"}}},{"_index" : "bank","_id" : "Galactic Banking Project-GBP-1","_score" : 0.5469647,"_source" : {"Issue" : {"summary" : "Development of API for International Transfers"}}}]}
}