Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 12:08
    jdrouet synchronize #9571
  • 11:57
    jdrouet edited #9665
  • 09:57
    jdrouet synchronize #9665
  • 07:19
    jdrouet opened #9665
  • 06:26
    jdrouet synchronize #9571
  • 05:12
    lucperkins edited #9664
  • 04:55
    lucperkins edited #9664
  • 04:55
    lucperkins opened #9664
  • 04:05
    dependabot[bot] labeled #9663
  • 04:04
    dependabot[bot] opened #9663
  • Oct 17 18:20
    mmacedoeu edited #9496
  • Oct 16 21:58
    lucperkins synchronize #9658
  • Oct 16 21:54
    lucperkins edited #9658
  • Oct 16 21:54
    lucperkins edited #9658
  • Oct 16 21:47
    lucperkins edited #9658
  • Oct 16 21:46
    lucperkins review_requested #9659
  • Oct 16 21:46
    lucperkins opened #9659
  • Oct 16 21:39
    lucperkins synchronize #9656
  • Oct 16 21:33
    lucperkins review_requested #9655
  • Oct 16 21:33
    lucperkins review_requested #9654
Mark Klass
@ChristianKlass
Hello, is there a way to use Vector's transformation to "clean" the fields? For example, I used the tokenize transformation to get some of the values from some log, but now, I have values like {"protocol":":udp", "source_port":":57714->", etc}
Is there a way to clean them? Like removing the : in protocol and the : and -> in source_port?
Bindu-Mawat
@Bindu-Mawat
Hi I am using http source and when I start and stop my application multiple times(which restart vector as well each time), on client side I start getting error in connecting to vector: Error message:Connection refused (Connection refused)
Ghost
@ghost~5cd19fe2d73408ce4fbfa0bf
i'm tringto use vecto on my docker compose
but it can't catch the traffic on 5000 port
on my local machine
Jesse Szwedko
@jszwedko
Hi @Bindu-Mawat and @mikele_g_twitter ; just a note that community discussion and support has moved to discord: https://discord.gg/jm97nzy
Vyacheslav Rakhinskiy
@rakhinskiy
HI, how I can disable rate_limits on errors (I set drop_invalid to true for json_parser)?
For example
Nov 25 12:11:45.976 WARN transform{name=nginx_parse_json type=json_parser}: vector::internal_events::json: 19 "Event failed to parse as JSON" events were rate limited. rate_limit_secs=5
Vyacheslav Rakhinskiy
@rakhinskiy
And how I can debug this json? I start vector with LOG="trace" and can see only WARN for this event
Jesse Szwedko
@jszwedko
Hi @rakhinskiy ! Just a note that community discussion and support has moved to discord: https://discord.gg/jm97nzy
Vyacheslav Rakhinskiy
@rakhinskiy
@jszwedko ok thanks
Liem Le Hoang Duc
@liemle3893

Hi there, I'm stuck at write log to file with correct timezone.
[sinks.all]
inputs = ["in"]
type = "file"
path = "/tmp/%Y-%m-%d/%H.log"
encoding.codec = "text"


The time is UTC based where I need it in Local Timezone (+7 in my case). Is there anyway to achieve this with Vector? I've searched around but no luck.

Jesse Szwedko
@jszwedko
Hi @liemle3893 ! Just a note that community discussion and support has moved to discord: https://discord.gg/jm97nzy
1 reply
valerypetrov
@valerypetrov
Hey, guys. Does the vector is fips compatible?
avalenn
@avalenn:matrix.org
[m]
Hi @valerypetrov ! Just a note that community discussion and support has moved to discord: https://discord.gg/jm97nzy
valerypetrov
@valerypetrov
Thanks
Alexandre
@akalend
Hi
I have error unknown field transform
config:
[transform.del_fields]
type = "remap"
inputs = ["import_logs_tr"]
source = '''
.del(.file, .host)
.log = "import"
'''
this transform is as in docs
data before transform is:
{"file":"/var/log/app/app.log","host":"244f68ac3445","message":"num **213","severity":"import","source_type":"file","timestamp":"2021-05-19T13:06:18Z"}
Sanskar Gupta
@Sanskar95

Hi
I have a rust tcp server running at 4000, when I try to start the vector to get the tcp metrics am getting

ERROR source{component_kind="source" component_name=my_source_id component_type=socket}: vector::sources::util::tcp: Failed to bind to listener socket. error=TCP bind failed: Address already in use (os error 98)
Jun 23 22:47:43.438 ERROR source{component_kind="source" component_name=my_source_id component_type=socket}: vector::topology: An error occurred that vector couldn't handle.
Here is my vector.toml

[sources.my_source_id]
type = "socket"
address = "0.0.0.0:4000"
mode = "tcp"

[sinks.out]
inputs = ["my_source_id"]
type = "console"
encoding = "text"

adamwolfe-tc
@adamwolfe-tc

anyone have an example of a config for getting pod logs from a specific kubernetes namespace only? i'm trying

sources:
  kube_log_source:
    type: kubernetes_logs
    extra_field_selector: metadata.namespace==vector-testing

transforms:
  filter_on_namespace:
    type: filter
    inputs:
      - kube_log_source
    condition: .kubernetes.namespace == "vector-testing"

to no avail

Pasha Radchenko
@ep4sh

Guys, could you please help me with the TOML configuration for vector?
I have the following configmap, trying to implement multiline for java stack traces:


 18     [transforms.my_custom_k8s_multiline]
 19     type = "reduce"
 20     inputs = [ "kuber" ]
 21     group_by = [ "file", "kubernetes", "source_type", "stream", "timestamp" ]
 22     starts_when.type = "vrl"
 23     starts_when.source = "match(.message, r'^\\[')"
 24     expire_after_ms = 3_000
 25     flush_period_ms = 300
 26
 27       [transforms.my_custom_k8s_multiline.merge_strategies]
 28       message = "concat_newline"

but it fails with something like

 ~ k logs vector-mx4h2
Sep 23 15:10:13.857  INFO vector::app: Log level is enabled. level="vector=info,codec=info,vrl=info,file_source=info,tower_limit=trace,rdkafka=info" enable_datadog_tracing=false
Sep 23 15:10:13.858  INFO vector::app: Loading configs. paths=["/etc/vector/vector.toml"]
Sep 23 15:10:13.862  INFO vector::sources::kubernetes_logs: Obtained Kubernetes Node name to collect logs for (self). self_node_name="ip-10-221-172-92.ec2.internal"
Sep 23 15:10:13.885  INFO vector::sources::kubernetes_logs: Excluding matching files. exclude_paths=["/var/log/pods/cluster-*/**", "/var/log/pods/kube-*/**", "/var/log/pods/*_nginx-*/**", "/var/log/pods/logging_*/**"]
Sep 23 15:10:13.912 ERROR vector::topology: Configuration error. error=Transform "my_custom_k8s_multiline":
error[E100]: unhandled error
  ┌─ :1:11 │ match(.message, r'^\[')
  │ ^^^^^^^^^^^^^^^^^^^^^^^
  │ │
  │ expression can result in runtime error
  │ handle the error case to ensure runtime success

I've tried a lot of variants with match function arguments, remove and revert quotes but still no luck. Really appreciate if somebody can look into :)

Ajay Gupta
@ajay978:matrix.org
[m]
Hi Can we write custom plugin in vector?