Been digging into this lately and figured I'd share, since it cleared up something I was fuzzy on: how BigQuery actually discovers Iceberg tables, and what's genuinely new here vs. what's just been repackaged.

The old ways of doing this:

1. Pointing BigQuery straight at a metadata file. Something like gs://bucket/orders/metadata/00003.metadata.json. It works, but it's fragile. Iceberg writes new metadata files instead of editing the old ones, so if any other engine commits a new snapshot, your pointer is instantly stale. You're stuck manually bumping it to 00004.metadata.json every time.
2. Catalog based access, like using AWS Glue as an external catalog. This actually fixes the staleness problem since the catalog tracks current state for you. But every engine still needed its own custom integration to talk to that catalog.

What's actually new: Google now has a managed Lakehouse Runtime Catalog (BigLake) that exposes a real Apache Iceberg REST Catalog endpoint, using the same open REST Catalog spec other engines already speak. So instead of building a one off integration per engine, anything that already understands Iceberg REST can just talk to BigLake directly. The data itself never moves, it stays in Cloud Storage. BigLake is really just a metadata and governance layer with IAM sitting on top of it.

There's also a version detail worth knowing: Iceberg 1.10 shipped a native BigQueryMetastoreCatalog and BigQueryMetastoreClient, plus a GoogleAuthManager that handles Google credential auth inside Iceberg's REST auth framework. It's easy to conflate these two things, but they're not the same. One's a catalog implementation built for BigQuery Metastore specifically, the other is the standardized REST interface. Google's steering people toward the REST endpoint for anything new where cross engine interoperability matters.

The post I read also gets into the practical setup side: creating a BigLake catalog (single bucket vs multi bucket), the IAM roles Google recommends (BigLake Admin, Editor, Viewer, plus Storage Object User), and shows an open source tool called OLake Go (full disclosure, I work on this) writing Iceberg tables directly into that catalog so BigQuery can see them right away, queryable through the usual four part PROJECT.CATALOG.NAMESPACE.TABLE identifier with no manual metadata pointer updates. There's a decent troubleshooting section too. Apparently if your connection test passes but the actual sync fails, it's almost always a missing storage level IAM role rather than bad credentials.

Full post here if you want the details: https://olake.io/blog/biglake-iceberg-rest-catalog-olake-go-setup/

Curious if anyone here has already moved existing Iceberg on BigQuery setups over to the REST catalog, or if you're all still doing the manual metadata file dance like I was.

Adding to the 10,000th - how do I become a cloud engineer?

I work as a VP and Information Security Officer at a bank (MS/Azure shop). I have my CISSP, CRISC, and CISA certifications. I have a bachelor's in Cyber and am working on my master's in Cyber now. I have long been interested in pivoting to a career in cyber. Every guide online says, "Learn the basics of Networks. Learn Linux. Learn Python/Bash/SQL. Pick a provider. Learn Terraform, CI/CD, DevOps. Build projects. Blog about it."

Is that really it? There must be a more structured approach, right? I have 6 years of experience in cybersecurity, but I am completely lost about how to target a cloud security role. I feel clueless. I'm currently under contract with my employer (they are paying for my master's), so I have about 24 months before I even start actively searching for a job. By then, will much of what I learn be obsolete? Do I take the standard path, or what would you recommend?d?

Hey everyone,

I'm looking for a cheap and reliable cloud provider for a non-production Kubernetes environment (mainly dev/testing/staging).

I don't need anything super powerful, just something reasonably priced and easy to manage.

What are you guys using for your non-prod environments? Any recommendations for something cheap with predictable pricing?

Would also be great to know roughly how much you're paying per month and what kind of workloads you're running.

Thanks!

During our shortlist process, we looked at a handful of platforms and compared them on this specific criterion:

Wiz, Orca, Prisma Cloud, Aqua Security, Lacework, CrowdStrikewere all part of our evaluation

Most of them push you toward their own proprietary AI layer rather than letting you plug in custom agents via MCP

Wasn't a dealbreaker at first, since it sounded like a minor technical detail early on

Became a real limitation across nearly every platform we tested once our team wanted to build custom detection workflows

how much weight other teams put on this versus more traditional factors like coverage and pricing.

i want to learn multicloud including aws ..azure..gcp..Huawei
is there any source or book to learn this ?

I'm looking for someone confident in the field to take some free time to just simplify things for me and help me understand the core necessities of the job.

Hello, I am giving myself a year and a half to study everything cloud computing: cloud engineering and devops mainly, to later add agentops and a little on data engineering.
I am currently using free tier AWS and have a free student Oracle account. I will later move onto Azure and end with GCP. I also have data bricks if it's ever useful.

I am taking free courses and YouTube university but the basic concepts are hard for me to understand and grasp fully. I feel when I know the basics, everything will fall into place.

I am fine with linux, but everything else makes little sense to me. I would describe it as learning a language and the function behind every language which I am struggling to grasp: scripting, APIs, etc.
I know a little about terraform but not at a confident level. I have no idea about K8s even after watching many videos on it. Postgres is basic SQL, so it's fine, I just need to refresh my knowledge. Github actions and other CI/CD, I have not gotten around to those yet.

I am studying CS (3 years in) but where I am studying it, a professional university at top in my school, isn't teaching it the best. AI helps but it can only do so much when the basics weren't taught to you properly.
I have already spent 3 months on it but I do not want to give up.

Please do not recommend courses on certs, I am doing that already.
Bonus points if you know enough about landing zones to teach for a short time.

I have no previous IT skills but plan to have that as my first role when leaving uni.

I am currently working as GWS administrator and I want to learn cloud and work as junior cloud engineer or whatever entry roles are available. So please tell me a roadmap to work on and which platform is better (AWS,azure,GCP)?

I’m currently working as an \\\*\\\*SAP Cloud Integration (CPI) consultant\\\*\\\* and considering moving into \\\*\\\*cloud-native integration technologies\\\*\\\* such as Azure Integration Services, AWS, or GCP.

Is this a good career move for someone with SAP CPI experience? How does the \\\*\\\*job market, salary potential, and long-term growth\\\*\\\* compare with staying in the SAP Integration ecosystem? Also, how transferable are CPI skills to these cloud platforms?

I’m trying to understand the industry’s use of AI in cloud/DevOps engineering roles. I am a cloud engineer and I use it daily for IaC development and CLI changes, I still review all changes manually but it’s just easier to do that vs write terraform from scratch, I analyze results in dev environment before making changes in prod.

I’m based in Italy and currently learning AWS, preparing for the **AWS Solutions Architect Associate**, and building projects with AWS almost every day. But I feel I’m missing something. I’d really appreciate any advice from people working in cloud recruitment:
**What skills are recruiters actually looking for in junior Cloud Support/Cloud Engineer candidates today? What should I learn beyond AWS?**

What **projects actually make a junior CV stand out**?
What skills should I prioritize and what exactly should I know: **Linux, networking, Python/Bash, Terraform, Docker, Kubernetes, CI/CD, Azure**?
What am I probably overlooking?
Are there **free resources, labs, certifications, communities or programs** you would strongly recommend?

If you’re a recruiter/hiring manager, what would make you choose one junior candidate over another?
I’m trying to avoid spending months learning things that don’t translate into employability. **Any concrete advice, resources, or even contacts are greatly appreciated.**

We've been on a leading CNAPP for about a year and started reevaluating this quarter, mostly driven by alert fatigue. A lot of "critical" findings turned out to be non-exploitable once we dug in, and after enough of those a couple of analysts started asking why we were paying for noise.

We've also been wanting more deployment flexibility since our current platform only offers SaaS or a pricier in-account option. That plus the alert fatigue is why we finally sat down and did a real Orca vs current platform comparison instead of just complaining about it in standups.

Spent about two weeks running it side-by-side, mostly focused on how each platform prioritized findings rather than just what they found, since raw detection counts weren't really the bottleneck for us. The prioritization gap was bigger than the coverage gap, wasn't expecting that.

If anyone here has actually gone through a full platform switch and whether the migration pain was worth it.