Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: object别名问题修复 #4583

Open
wants to merge 36 commits into
base: deploy/paas3-dev-1217
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
ef1a44a
feat: 功能拉取
jidanjuan Dec 5, 2024
bdc4890
feat: 功能拉取
jidanjuan Dec 5, 2024
71e6819
feat: 字段分析调整& echarts升级
jidanjuan Dec 6, 2024
4714a61
feat: 别名切换功能&对接别名的接口
jidanjuan Dec 9, 2024
57b882e
feat: 字段分析列表下载完成
jidanjuan Dec 10, 2024
abf500c
feat: ext字段做特殊的展示 30%
jidanjuan Dec 10, 2024
a0cd4f0
feat: ext字段展开切换功能
jidanjuan Dec 12, 2024
c0c6f8e
feat: ext字段展开切换功能
jidanjuan Dec 12, 2024
3c799fa
feat: 字段配置中,ext字段做树状结构展示;
jidanjuan Dec 12, 2024
15658b1
feat: 清洗字段校验逻辑调整和增加别名功能
jidanjuan Dec 13, 2024
e215258
feat: 字段列表支持object 格式字段的层级展示
jidanjuan Dec 16, 2024
427fe5d
feat: 字段列表支持object 格式字段的层级展示
jidanjuan Dec 16, 2024
ed2612b
feat: 字段列表支持object 格式字段的层级展示
jidanjuan Dec 16, 2024
cd3c514
feat: 字段列表支持object 格式字段的层级展示
jidanjuan Dec 17, 2024
2d82035
feat: 字段列表支持object 格式字段的层级展示
jidanjuan Dec 17, 2024
b869fab
feat: 删除多余打印
jidanjuan Dec 17, 2024
8c570cd
feat: 别名和object分割点
jidanjuan Dec 18, 2024
6cdf371
feat: 别名提交逻辑修改
jidanjuan Dec 18, 2024
022cc0b
Merge branch master of https://github.com/TencentBlueKing/bk-monitor …
jidanjuan Dec 18, 2024
33429c2
Merge branch master of https://github.com/TencentBlueKing/bk-monitor …
jidanjuan Dec 18, 2024
cf0f1b3
feat: 字段清洗支持别名修改和object字段
jidanjuan Dec 20, 2024
e32a431
feat: 字段清洗支持别名修改和object字段
jidanjuan Dec 23, 2024
5226287
feat: 字段清洗支持别名修改和object字段
jidanjuan Dec 23, 2024
2f5f6cd
feat: 编辑清洗保存逻辑修改
jidanjuan Dec 23, 2024
16f2c58
feat: 编辑清洗保存逻辑修改
jidanjuan Dec 23, 2024
9eae273
fix: object别名问题修复
jidanjuan Dec 24, 2024
9340edf
fix: object别名问题修复
jidanjuan Dec 24, 2024
0cd1e37
fix: object别名问题修复
jidanjuan Dec 25, 2024
b1c69e7
fix: object别名问题修复
jidanjuan Dec 25, 2024
5eeeb88
Merge branch 'merge/field-analysis' of https://github.com/jidanjuan/b…
jidanjuan Dec 25, 2024
99bc115
Merge branch 'merge/field-analysis' of https://github.com/jidanjuan/b…
jidanjuan Dec 25, 2024
2929d31
fix: object别名问题修复
jidanjuan Dec 26, 2024
5e4b21f
fix: object别名问题修复
jidanjuan Dec 26, 2024
cdb0304
fix: object别名问题修复
jidanjuan Dec 27, 2024
92a9db0
fix: 下载格式修改补充
jidanjuan Dec 27, 2024
e1f6a93
fix: object别名问题修复
jidanjuan Dec 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
247 changes: 148 additions & 99 deletions bklog/web/src/components/collection-access/field-table.vue

Large diffs are not rendered by default.

59 changes: 44 additions & 15 deletions bklog/web/src/components/collection-access/step-field.vue
Original file line number Diff line number Diff line change
Expand Up @@ -1028,6 +1028,7 @@
is_delete: false,
is_dimension: false,
is_time: false,
query_alias:'',
value: '',
option: {
time_format: '',
Expand Down Expand Up @@ -1087,9 +1088,11 @@
field_name: '',
field_type: '',
description: '',
query_alias:'',
is_case_sensitive: false,
is_analyzed: false,
is_built_in: false,
is_add_in: true,
is_dimension: false,
previous_type: '',
tokenize_on_chars: '',
Expand All @@ -1115,7 +1118,8 @@
metaDataList: [],
isDebugLoading: false,
builtFieldShow:false,
fieldsObjectData: []
fieldsObjectData: [],
alias_settings:[]
};
},
computed: {
Expand Down Expand Up @@ -1324,18 +1328,20 @@
this.formData.fields = [... this.formData.fields,...this.copyBuiltField]
this.savaFormData();
}else{
const allFields = this.$refs.fieldTable.getData();
const copyBuiltFieldIds = new Set(this.copyBuiltField.map(field => field.field_name));
const { copyFields, remainingFields } = allFields.reduce((acc, field) => {
if (copyBuiltFieldIds.has(field.field_name)) {
acc.copyFields.push(field);
} else {
acc.remainingFields.push(field);
const allFields = this.$refs.fieldTable.getData();
const copyBuiltFieldIds = new Set(this.copyBuiltField.map(field => field.field_name));
const { copyFields, remainingFields } = allFields.reduce((acc, field) => {
if (copyBuiltFieldIds.has(field.field_name)) {
acc.copyFields.push(field);
} else {
acc.remainingFields.push(field);
}
return acc;
}, { copyFields: [], remainingFields: [] });
this.formData.fields = remainingFields;
if(copyFields.length){
this.copyBuiltField = copyFields;
}
return acc;
}, { copyFields: [], remainingFields: [] });
this.formData.fields = remainingFields;
this.copyBuiltField = copyFields;
}
},
// 初始化清洗项
Expand Down Expand Up @@ -1897,7 +1903,6 @@
row.option = Object.assign({}, option);
}
});

this.params.etl_config = etl_config;
Object.assign(this.params.etl_params, {
separator_regexp: etlParams?.separator_regexp || '',
Expand Down Expand Up @@ -2284,6 +2289,7 @@
.then(res => {
if (res.data) {
const { clean_type, etl_params: etlParams, etl_fields: etlFields } = res.data;
this.concatenationQueryAlias(etlFields)
this.formData.fields.splice(0, this.formData.fields.length);

this.params.etl_config = clean_type;
Expand Down Expand Up @@ -2345,8 +2351,6 @@
},
// 新建、编辑采集项时获取更新详情
async setDetail(id) {
console.log(23333);

if (!id) return;
this.basicLoading = true;
this.$http
Expand All @@ -2355,6 +2359,15 @@
})
.then(async res => {
if (res.data) {
let keys = Object.keys(res.data.alias_settings);
let arr = keys.map( key => {
return {
query_alias : key,
field_name : res.data.alias_settings[key].path
}
})
this.alias_settings = arr
this.concatenationQueryAlias( res.data.fields)
this.$store.commit('collect/setCurCollect', res.data);
this.getDetail();
await this.getCleanStash(id);
Expand All @@ -2366,6 +2379,16 @@
this.basicLoading = false;
});
},
// 拼接query_alias
concatenationQueryAlias(fields) {
fields.forEach(item => {
this.alias_settings.forEach(item2 => {
if( item.field_name === item2.field_name || item.alias_name === item2.field_name ){
item.query_alias = item2.query_alias
}
})
})
},
// 新增、编辑清洗选择采集项
async handleCollectorChange(id) {
this.basicLoading = true;
Expand Down Expand Up @@ -2562,6 +2585,9 @@
},
/** 获取fields */
async requestFields(indexSetId) {
const typeConversion= {
keyword : 'string'
}
try {
const res = await this.$http.request('retrieve/getLogTableHead', {
params: {
Expand All @@ -2571,6 +2597,8 @@
this.fieldsObjectData = res.data.fields.filter(item => item.field_name.includes('.'))
this.fieldsObjectData.forEach(item => {
let name = item.field_name.split('.')[0]
item.field_type = typeConversion[item.field_type]
item.is_objectKey = true
this.copyBuiltField.forEach( builtField => {
if(builtField.field_type === "object" && name.includes(builtField.field_name)){
if (!Array.isArray(builtField.children)) {
Expand All @@ -2581,6 +2609,7 @@
}
} )
})

} catch (err) {
console.warn(err);
}
Expand Down
23 changes: 22 additions & 1 deletion bklog/web/src/global/field-setting.vue
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,7 @@
participleState: 'default',
is_edit: true,
});
const alias_settings = ref([])
const batchAddField = () => {
console.log(collectorConfigId.value, 'collectorConfigId');
if (!collectorConfigId.value) return;
Expand Down Expand Up @@ -442,6 +443,15 @@
},
})
.then(res => {
let keys = Object.keys(res.data.alias_settings);
let arr = keys.map( key => {
return {
query_alias : key,
field_name : res.data.alias_settings[key].path
}
})
alias_settings.value = arr

const collectData = res?.data || {};
formData.value = collectData;
cleanType.value = collectData?.etl_config;
Expand All @@ -456,6 +466,14 @@
},
})
.then(res => {
console.log(res);
res.data.etl_fields.forEach(item => {
alias_settings.value.forEach(item2 => {
if( item.field_name === item2.field_name || item.alias_name === item2.field_name ){
item.query_alias = item2.query_alias
}
})
})
tableField.value = res?.data?.etl_fields.filter(item => !item.is_built_in && !item.is_delete);
formData.value.etl_params.retain_original_text = res?.data?.etl_params.retain_original_text;
});
Expand Down Expand Up @@ -529,12 +547,15 @@
alias_settings: [
...indexfieldTableData.map(item =>{
return {
field_name: item.field_name,
field_name: item.alias_name || item.field_name,
query_alias: item.query_alias,
path_type: item.field_type}
}),
],
};
console.log(data);

return
await http
.request('collect/fastUpdateCollection', {
params: {
Expand Down
74 changes: 59 additions & 15 deletions bklog/web/src/global/setting-table.vue
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,10 @@
size="small"
class="expand-table"
col-border>
<bk-table-column label="字段名" prop="field_name" width="180">
<bk-table-column label="字段名" prop="field_name" width="220">
<template #default="props">
<div class="ext-field_name">
<span class="ext-subnode bklog-icon bklog-subnode"></span>
<span v-bk-tooltips.top="props.row.field_name">{{ props.row.field_name }}</span>
</div>
</template>
Expand Down Expand Up @@ -96,7 +97,7 @@
</bk-form-item>
</template>
</bk-table-column>
<bk-table-column label="数据类型" prop="field_type" width="120" align="center"></bk-table-column>
<bk-table-column label="数据类型" prop="field_type" width="100" align="center"></bk-table-column>
<bk-table-column label="分词符" prop="" width="200"></bk-table-column>
</bk-table>
</template>
Expand All @@ -106,17 +107,31 @@
<bk-table-column
:render-header="renderHeaderFieldName"
:resizable="false"
width="180"
width="220"
>
<template #default="props">
<div
v-if="!props.row.is_edit"
class="overflow-tips"
class="field-name-overflow-tips"
v-bk-overflow-tips
>
<span v-if="props.row.field_name === 'ext' && !extExpand" @click="expandObject(props.row,true)" class="ext-btn bk-icon icon-angle-right"></span>
<span v-if="props.row.field_name === 'ext' && extExpand" @click="expandObject(props.row,false)" class="ext-btn bk-icon icon-angle-down"></span>
<span v-bk-tooltips.top="$t('字段名不支持快速修改')">{{ props.row.field_name }} </span>
<span v-if="props.row.field_name === 'ext' && !extExpand" @click="expandObject(props.row,true)" class="ext-btn rotate bklog-icon bklog-arrow-down-filled"></span>
<span v-if="props.row.field_name === 'ext' && extExpand" @click="expandObject(props.row,false)" class="ext-btn bklog-icon bklog-arrow-down-filled"></span>
<div v-if="!props.row.alias_name" v-bk-tooltips.top="$t('字段名不支持快速修改')" class="field-name">{{ props.row.field_name }} </div>
<div v-else class="field-name-box">
<div class="alias-name">{{ props.row.field_name }}</div>
<div
class="participle-icon"
:class="getFieldEditDisabled(props.row)?'participle-icon-color':''"
>
<i
style ='color: #3A84FF;margin: 0 10px;'
class="bk-icon bklog-icon bklog-yingshe"
></i>
</div>
<div class="alias-name" v-if="isPreviewMode || props.row.is_built_in">{{ props.row.alias_name}}</div>
<bk-input class="alias-name" v-else v-model.trim="props.row.alias_name"></bk-input>
</div>
</div>
<bk-form-item
v-else
Expand Down Expand Up @@ -151,14 +166,14 @@
class="overflow-tips"
v-bk-overflow-tips
>
<span>{{ props.row.alias_name }}</span>
<span>{{ props.row.query_alias }}</span>
</div>
<bk-form-item
v-else
:class="{ 'is-required is-error': props.row.aliasErr }"
>
<bk-input
v-model.trim="props.row.alias_name"
v-model.trim="props.row.query_alias"
:disabled="props.row.is_delete || isSetDisabled"
@blur="checkAliasNameItem(props.row)"
>
Expand All @@ -178,7 +193,7 @@
:render-header="renderHeaderDataType"
:resizable="false"
align="center"
width="120"
width="100"
>
<template #default="props">
<div
Expand Down Expand Up @@ -470,7 +485,7 @@
// trigger: 'blur'
// }
],
alias_name: [
query_alias: [
// 目前组件不能拿到其他字段的值,不能通过validator进行验证
// {
// validator: this.checkAliasName,
Expand Down Expand Up @@ -527,7 +542,7 @@
if (this.keyword) {
const query = this.keyword.toLowerCase();
return currentTableList.filter(
item => item.field_name.toLowerCase().includes(query) || item.alias_name.toLowerCase().includes(query),
item => item.field_name.toLowerCase().includes(query) || item.query_alias.toLowerCase().includes(query),
);
} else {
return currentTableList;
Expand Down Expand Up @@ -820,7 +835,7 @@
});
},
checkAliasNameItem(row) {
const { field_name: fieldName, alias_name: aliasName, is_delete: isDelete } = row;
const { field_name: fieldName, query_alias: aliasName, is_delete: isDelete } = row;
if (isDelete) {
return true;
}
Expand Down Expand Up @@ -898,7 +913,7 @@
directives: [
{
name: 'bk-tooltips',
value: this.$t('非必填字段,填写后将会替代字段名;字段名与内置字段重复时,必须重新命名。'),
value: this.$t('填写后原字段名和别名均可查询'),
},
],
class: 'render-header decoration-header-cell',
Expand Down Expand Up @@ -1045,6 +1060,9 @@
}
:deep(.ext-field_name) {
margin-left: 20px;
.ext-subnode{
font-size: 16px;
}
}
}
:deep(.bk-table-body) {
Expand All @@ -1061,12 +1079,38 @@

.overflow-tips {
padding: 10px 15px;
}
.field-name-overflow-tips{
.ext-btn{
cursor: pointer;
font-size: 18px;
position: absolute;
left: 0;
bottom: 10px;
}
.rotate{
transform: rotate(-90deg);
}
.field-name{
margin: 15px 15px;
}
.field-name-box{
display: flex;
height: 100%;
align-items: center;
.alias-name{
padding-left: 15px;
width: 50%
}
.participle-icon{
font-size: 18px;
left: 40%;
width: 10%;
position: absolute;
z-index: 999
}
.participle-icon-color{
background-color: rgb(250, 251, 253) !important;
}
}
}
}
Expand Down
1 change: 1 addition & 0 deletions bklog/web/src/language/lang/en/button.ts
Original file line number Diff line number Diff line change
Expand Up @@ -210,4 +210,5 @@ export default {
取消全屏: 'Exit Full Screen',
展示字段名: 'Show Field Name',
展示别名: 'Show Alias Name',
内置: 'built-in',
};
6 changes: 4 additions & 2 deletions bklog/web/src/language/lang/en/content.ts
Original file line number Diff line number Diff line change
Expand Up @@ -498,8 +498,8 @@ export default {
'You currently do not have the business--${0} permission, please contact the operation and maintenance classmate {1} to add the role',
'聚类字段的最大长度,如果超过这个长度将直接丢弃,设置越大将消耗更多的资源':
'The maximum length of the clustering field. if it exceeds this length, it will be directly discarded. the larger the setting, the more resources it will consume.',
'非必填字段,填写后将会替代字段名;字段名与内置字段重复时,必须重新命名。':
'Non-required fields, after filling in, will replace the field name; when the field name conflicts with the built-in field, it must be renamed.',
'填写后原字段名和别名均可查询':
'The original field name and the alias can both be used for querying after being populated.',
'只能基于一个字段进行聚类,并且字段是为text的分词类型,默认为log字段':
"Clustering can only be based on one field, and the field must be of text tokenization type, defaulting to 'log' field",
' - 日志内容:[2006-01-02 15:04:05] content': ' - Log content: [2006-01-02 15:04:05] content',
Expand Down Expand Up @@ -998,4 +998,6 @@ export default {
'当前缺少指标': 'Current metrics are missing',
字段名不支持快速修改: 'Field name does not support quick modification',
去重后字段统计: 'Field statistics after deduplication',
"检测到字段名称包含异常值,只能包含a-z、A-Z、0-9和_,且不能以_开头和结尾。请重命名,命名后原字段将被覆盖;": 'The field name contains invalid characters. It can only include a-z, A-Z, 0-9, and _, and it cannot start or end with _. Please rename it; the original field will be overwritten after renaming',
"检测到字段名与系统内置名称冲突。请重命名,命名后原字段将被覆盖": 'The field name conflicts with a system built-in name. Please rename it; the original field will be overwritten after renaming',
};
Loading
Loading